Project Title : Trend Minder Stock Predictor (TMSP)
Team Members: Amit Sai Jitta and Hemraj Yadav
Yahoo Finance: https://en.wikipedia.org/wiki/List_of_S%26P_500_companies
Dataset:
1569753 entries, 2020-01-02 to 2024-02-29
Our project dataset contains 1569753 rows with comprehensive financial information for companies listed on the S&P 500 index over a period of four years. It includes data points such as stock prices, trading volume, companies’ names etc.
Scope: All companies listed on the S&P 500 index
Duration: 4 years
Frequency: Daily (or as per your dataset's granularity)
Data Format: Tabular data, likely organized in a structured format such as CSV (Comma Separated Values)
Variables:
Date: specific calendar date on which a stock market transaction occurred, Data Type - Date (mm-dd-yyyy)
Open: price at which a particular stock begins trading on a given trading day, Data Type - floating
Close: final price at which a particular stock is traded on a given trading day, Data Type - floating
high: highest price at which a particular stock traded during a given trading day, Data Type - floating.
low: lowest price at which a particular stock traded during a given trading day, Data Type- floating.
Volume: total number of shares of a particular stock that are traded during a specified period, Data Type - Integer
Name: Unique identifier or symbol used to represent a particular publicly traded company on a stock exchange, Data Type – Character
Origin and Purpose:
Wiki collected the dataset with the purpose of conducting intriguing visualizations on stock market volatility. It has been made available for others to delve into the intricacies of the stock market domain and to engage in various data visualization exercises. One can look at simple things like how prices change over time, graph, and compare multiple stocks at once, or generate and graph new metrics from the data provided.From these data informative stock stats such as volatility and moving averages can be easily calculated.
Data Collection:
Gather relevant time series data from appropriate sources. Ensure data quality and consistency.
Data Preprocessing:
Handle missing values, outliers, and noise in the data. Perform any necessary data transformations (e.g., normalization, scaling). Consider techniques like detrending or deseasonalizing if needed.
Exploratory Data Analysis (EDA):
Understand the underlying patterns, trends, and seasonality in the data. Visualize the time series data to gain insights. Identify potential relationships and correlations with other variables.
Feature Engineering:
Derive relevant features from the time series data. Consider lagged variables, rolling statistics, or domain-specific features. Feature selection may be necessary to reduce dimensionality.
Model Selection:
Choose appropriate time series models based on the problem and data characteristics. Consider traditional statistical models (e.g., ARIMA, SARIMA) or machine learning models (e.g., LSTM, Prophet). Evaluate the trade-offs between complexity and interpretability.
Model Training:
Split the data into training and validation/test sets. Train the selected models using the training data. Tune model hyperparameters to optimize performance.
Model Evaluation:
Evaluate model performance using appropriate metrics (e.g., Mean Absolute Error, Root Mean Squared Error). Compare the performance of different models. Validate the models using the validation/test set.
Model Interpretation:
Interpret the results of the trained models. Understand the impact of different features on the predictions. Identify any limitations or assumptions of the models.
Deployment and Monitoring:
Deploy the chosen model into production if applicable. Set up monitoring systems to track model performance over time. Implement feedback loops to update the model as new data becomes available.
Documentation and Reporting:
Document the entire process including data sources, preprocessing steps, model selection, and evaluation. Prepare a report summarizing the findings and recommendations. Communicate results effectively to stakeholders.
Iterative Improvement:
Continuously refine the model based on feedback and new data. Explore advanced techniques or alternative models to improve performance.
from google.colab import drive
drive.mount('/content/drive')
!pip install fasteda -q
Preparing metadata (setup.py) ... done Building wheel for fasteda (setup.py) ... done
More information on the fasteda package.
import pandas as pd
import missingno as msno
import matplotlib.pyplot as plt
import seaborn as sns
import os
import numpy as np
from fasteda import fast_eda
import plotly.express as px
from statsmodels.tsa.seasonal import seasonal_decompose
from statsmodels.tsa.stattools import adfuller
from statsmodels.tsa.stattools import acf
from statsmodels.tsa.statespace.sarimax import SARIMAX
from statsmodels.tsa.holtwinters import ExponentialSmoothing
from statsmodels.stats.diagnostic import acorr_ljungbox
from scipy.stats import shapiro
from sklearn.metrics import mean_absolute_error, mean_squared_error
from pandas import read_csv
from matplotlib import pyplot
from pandas import Series
import warnings
from pandas import DataFrame
from statsmodels.tsa.arima.model import ARIMA
from statsmodels.graphics.tsaplots import plot_acf
from statsmodels.graphics.tsaplots import plot_pacf
from scipy.stats import boxcox
from statsmodels.graphics.gofplots import qqplot
from sklearn.metrics import mean_squared_error
from math import sqrt
from math import exp
from math import log
import numpy
from statsmodels.tsa.arima.model import ARIMAResults
import yfinance as yf
pip install yfinance
Requirement already satisfied: yfinance in /usr/local/lib/python3.10/dist-packages (0.2.37) Requirement already satisfied: pandas>=1.3.0 in /usr/local/lib/python3.10/dist-packages (from yfinance) (1.5.3) Requirement already satisfied: numpy>=1.16.5 in /usr/local/lib/python3.10/dist-packages (from yfinance) (1.25.2) Requirement already satisfied: requests>=2.31 in /usr/local/lib/python3.10/dist-packages (from yfinance) (2.31.0) Requirement already satisfied: multitasking>=0.0.7 in /usr/local/lib/python3.10/dist-packages (from yfinance) (0.0.11) Requirement already satisfied: lxml>=4.9.1 in /usr/local/lib/python3.10/dist-packages (from yfinance) (4.9.4) Requirement already satisfied: appdirs>=1.4.4 in /usr/local/lib/python3.10/dist-packages (from yfinance) (1.4.4) Requirement already satisfied: pytz>=2022.5 in /usr/local/lib/python3.10/dist-packages (from yfinance) (2023.4) Requirement already satisfied: frozendict>=2.3.4 in /usr/local/lib/python3.10/dist-packages (from yfinance) (2.4.0) Requirement already satisfied: peewee>=3.16.2 in /usr/local/lib/python3.10/dist-packages (from yfinance) (3.17.1) Requirement already satisfied: beautifulsoup4>=4.11.1 in /usr/local/lib/python3.10/dist-packages (from yfinance) (4.12.3) Requirement already satisfied: html5lib>=1.1 in /usr/local/lib/python3.10/dist-packages (from yfinance) (1.1) Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4>=4.11.1->yfinance) (2.5) Requirement already satisfied: six>=1.9 in /usr/local/lib/python3.10/dist-packages (from html5lib>=1.1->yfinance) (1.16.0) Requirement already satisfied: webencodings in /usr/local/lib/python3.10/dist-packages (from html5lib>=1.1->yfinance) (0.5.1) Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas>=1.3.0->yfinance) (2.8.2) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31->yfinance) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31->yfinance) (3.6) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31->yfinance) (2.0.7) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31->yfinance) (2024.2.2)
# Download S&P 500 data from Wikipedia
def get_sp500_tickers():
sp500_df = pd.read_html("https://en.wikipedia.org/wiki/List_of_S%26P_500_companies")[0]
sp500_tickers = sp500_df['Symbol'].tolist()
return sp500_tickers
sp500_tickers = get_sp500_tickers()
print(sp500_tickers)
['MMM', 'AOS', 'ABT', 'ABBV', 'ACN', 'ADBE', 'AMD', 'AES', 'AFL', 'A', 'APD', 'ABNB', 'AKAM', 'ALB', 'ARE', 'ALGN', 'ALLE', 'LNT', 'ALL', 'GOOGL', 'GOOG', 'MO', 'AMZN', 'AMCR', 'AEE', 'AAL', 'AEP', 'AXP', 'AIG', 'AMT', 'AWK', 'AMP', 'AME', 'AMGN', 'APH', 'ADI', 'ANSS', 'AON', 'APA', 'AAPL', 'AMAT', 'APTV', 'ACGL', 'ADM', 'ANET', 'AJG', 'AIZ', 'T', 'ATO', 'ADSK', 'ADP', 'AZO', 'AVB', 'AVY', 'AXON', 'BKR', 'BALL', 'BAC', 'BK', 'BBWI', 'BAX', 'BDX', 'BRK.B', 'BBY', 'BIO', 'TECH', 'BIIB', 'BLK', 'BX', 'BA', 'BKNG', 'BWA', 'BXP', 'BSX', 'BMY', 'AVGO', 'BR', 'BRO', 'BF.B', 'BLDR', 'BG', 'CDNS', 'CZR', 'CPT', 'CPB', 'COF', 'CAH', 'KMX', 'CCL', 'CARR', 'CTLT', 'CAT', 'CBOE', 'CBRE', 'CDW', 'CE', 'COR', 'CNC', 'CNP', 'CF', 'CHRW', 'CRL', 'SCHW', 'CHTR', 'CVX', 'CMG', 'CB', 'CHD', 'CI', 'CINF', 'CTAS', 'CSCO', 'C', 'CFG', 'CLX', 'CME', 'CMS', 'KO', 'CTSH', 'CL', 'CMCSA', 'CMA', 'CAG', 'COP', 'ED', 'STZ', 'CEG', 'COO', 'CPRT', 'GLW', 'CTVA', 'CSGP', 'COST', 'CTRA', 'CCI', 'CSX', 'CMI', 'CVS', 'DHR', 'DRI', 'DVA', 'DAY', 'DE', 'DAL', 'XRAY', 'DVN', 'DXCM', 'FANG', 'DLR', 'DFS', 'DG', 'DLTR', 'D', 'DPZ', 'DOV', 'DOW', 'DHI', 'DTE', 'DUK', 'DD', 'EMN', 'ETN', 'EBAY', 'ECL', 'EIX', 'EW', 'EA', 'ELV', 'LLY', 'EMR', 'ENPH', 'ETR', 'EOG', 'EPAM', 'EQT', 'EFX', 'EQIX', 'EQR', 'ESS', 'EL', 'ETSY', 'EG', 'EVRG', 'ES', 'EXC', 'EXPE', 'EXPD', 'EXR', 'XOM', 'FFIV', 'FDS', 'FICO', 'FAST', 'FRT', 'FDX', 'FIS', 'FITB', 'FSLR', 'FE', 'FI', 'FLT', 'FMC', 'F', 'FTNT', 'FTV', 'FOXA', 'FOX', 'BEN', 'FCX', 'GRMN', 'IT', 'GEHC', 'GEN', 'GNRC', 'GD', 'GE', 'GIS', 'GM', 'GPC', 'GILD', 'GPN', 'GL', 'GS', 'HAL', 'HIG', 'HAS', 'HCA', 'DOC', 'HSIC', 'HSY', 'HES', 'HPE', 'HLT', 'HOLX', 'HD', 'HON', 'HRL', 'HST', 'HWM', 'HPQ', 'HUBB', 'HUM', 'HBAN', 'HII', 'IBM', 'IEX', 'IDXX', 'ITW', 'ILMN', 'INCY', 'IR', 'PODD', 'INTC', 'ICE', 'IFF', 'IP', 'IPG', 'INTU', 'ISRG', 'IVZ', 'INVH', 'IQV', 'IRM', 'JBHT', 'JBL', 'JKHY', 'J', 'JNJ', 'JCI', 'JPM', 'JNPR', 'K', 'KVUE', 'KDP', 'KEY', 'KEYS', 'KMB', 'KIM', 'KMI', 'KLAC', 'KHC', 'KR', 'LHX', 'LH', 'LRCX', 'LW', 'LVS', 'LDOS', 'LEN', 'LIN', 'LYV', 'LKQ', 'LMT', 'L', 'LOW', 'LULU', 'LYB', 'MTB', 'MRO', 'MPC', 'MKTX', 'MAR', 'MMC', 'MLM', 'MAS', 'MA', 'MTCH', 'MKC', 'MCD', 'MCK', 'MDT', 'MRK', 'META', 'MET', 'MTD', 'MGM', 'MCHP', 'MU', 'MSFT', 'MAA', 'MRNA', 'MHK', 'MOH', 'TAP', 'MDLZ', 'MPWR', 'MNST', 'MCO', 'MS', 'MOS', 'MSI', 'MSCI', 'NDAQ', 'NTAP', 'NFLX', 'NEM', 'NWSA', 'NWS', 'NEE', 'NKE', 'NI', 'NDSN', 'NSC', 'NTRS', 'NOC', 'NCLH', 'NRG', 'NUE', 'NVDA', 'NVR', 'NXPI', 'ORLY', 'OXY', 'ODFL', 'OMC', 'ON', 'OKE', 'ORCL', 'OTIS', 'PCAR', 'PKG', 'PANW', 'PARA', 'PH', 'PAYX', 'PAYC', 'PYPL', 'PNR', 'PEP', 'PFE', 'PCG', 'PM', 'PSX', 'PNW', 'PXD', 'PNC', 'POOL', 'PPG', 'PPL', 'PFG', 'PG', 'PGR', 'PLD', 'PRU', 'PEG', 'PTC', 'PSA', 'PHM', 'QRVO', 'PWR', 'QCOM', 'DGX', 'RL', 'RJF', 'RTX', 'O', 'REG', 'REGN', 'RF', 'RSG', 'RMD', 'RVTY', 'RHI', 'ROK', 'ROL', 'ROP', 'ROST', 'RCL', 'SPGI', 'CRM', 'SBAC', 'SLB', 'STX', 'SRE', 'NOW', 'SHW', 'SPG', 'SWKS', 'SJM', 'SNA', 'SO', 'LUV', 'SWK', 'SBUX', 'STT', 'STLD', 'STE', 'SYK', 'SYF', 'SNPS', 'SYY', 'TMUS', 'TROW', 'TTWO', 'TPR', 'TRGP', 'TGT', 'TEL', 'TDY', 'TFX', 'TER', 'TSLA', 'TXN', 'TXT', 'TMO', 'TJX', 'TSCO', 'TT', 'TDG', 'TRV', 'TRMB', 'TFC', 'TYL', 'TSN', 'USB', 'UBER', 'UDR', 'ULTA', 'UNP', 'UAL', 'UPS', 'URI', 'UNH', 'UHS', 'VLO', 'VTR', 'VLTO', 'VRSN', 'VRSK', 'VZ', 'VRTX', 'VFC', 'VTRS', 'VICI', 'V', 'VMC', 'WRB', 'WAB', 'WBA', 'WMT', 'DIS', 'WBD', 'WM', 'WAT', 'WEC', 'WFC', 'WELL', 'WST', 'WDC', 'WRK', 'WY', 'WHR', 'WMB', 'WTW', 'GWW', 'WYNN', 'XEL', 'XYL', 'YUM', 'ZBRA', 'ZBH', 'ZION', 'ZTS']
# Replace specified tickers with their replacements
tickers_to_replace = {"BF.B": "BF-B", "BRK.B": "BRK-B"}
for old_ticker, new_ticker in tickers_to_replace.items():
if old_ticker in sp500_tickers:
index = sp500_tickers.index(old_ticker)
sp500_tickers[index] = new_ticker
# Loop through each ticker and retrieve historical data
for ticker in sp500_tickers:
data = yf.download(ticker, start="2020-01-01", end="2024-03-01")
data.to_csv("/content/drive/MyDrive/data/tmsp2/" + f"{ticker}_historical_data.csv")
[*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed [*********************100%%**********************] 1 of 1 completed
# Combined the data containing the individual CSV files to a new CSV file
directory_path = "/content/drive/MyDrive/data/tmsp2/"
combined_data = pd.DataFrame()
for filename in os.listdir(directory_path):
if filename.endswith(".csv"):
filepath = os.path.join(directory_path, filename)
data = pd.read_csv(filepath)
ticker = filename.split("_")[0]
data["Ticker"] = ticker
combined_data = combined_data.append(data, ignore_index=True)
combined_data.to_csv("/content/drive/MyDrive/data/tmsp2/combined_sp500_data.csv", index=False)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True) <ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. combined_data = combined_data.append(data, ignore_index=True)
# Display the first few rows of the DataFrame
combined_csv_path = "/content/drive/MyDrive/data/tmsp2/combined_sp500_data.csv"
combined_data = pd.read_csv(combined_csv_path)
print(combined_data.head())
Date Open High Low Close Adj Close \
0 2020-01-02 177.679993 180.009995 177.139999 180.000000 149.672821
1 2020-01-03 177.020004 178.660004 175.630005 178.449997 148.383957
2 2020-01-06 177.149994 178.710007 176.350006 178.619995 148.525299
3 2020-01-07 178.279999 178.509995 176.820007 177.899994 147.926605
4 2020-01-08 178.000000 181.500000 177.649994 180.630005 150.196655
Volume Ticker
0 3601700 MMM
1 2466900 MMM
2 1998000 MMM
3 2173000 MMM
4 2758300 MMM
combined_data['date']=pd.to_datetime(combined_data['Date'])
combined_data['Date'] = pd.to_datetime(combined_data['Date'])
combined_data.set_index('Date', inplace=True)
# Display the summary information of the DataFrame
print("Summary Information:")
print(combined_data.info())
print("\nSummary Statistics:")
print(combined_data.describe())
Summary Information:
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 1569753 entries, 2020-01-02 to 2024-02-29
Data columns (total 8 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 Open 1569753 non-null float64
1 High 1569753 non-null float64
2 Low 1569753 non-null float64
3 Close 1569753 non-null float64
4 Adj Close 1569753 non-null float64
5 Volume 1569753 non-null int64
6 Ticker 1569753 non-null object
7 date 1569753 non-null datetime64[ns]
dtypes: datetime64[ns](1), float64(5), int64(1), object(1)
memory usage: 107.8+ MB
None
Summary Statistics:
Open High Low Close Adj Close \
count 1.569753e+06 1.569753e+06 1.569753e+06 1.569753e+06 1.569753e+06
mean 1.646755e+02 1.667576e+02 1.625807e+02 1.647067e+02 1.605460e+02
std 2.926456e+02 2.963269e+02 2.891349e+02 2.928045e+02 2.923336e+02
min 3.220000e+00 3.290000e+00 3.020000e+00 3.120000e+00 2.957920e+00
25% 5.543000e+01 5.617000e+01 5.468000e+01 5.542000e+01 5.218929e+01
50% 1.008500e+02 1.022100e+02 9.951000e+01 1.008700e+02 9.659942e+01
75% 1.903600e+02 1.927500e+02 1.879800e+02 1.904300e+02 1.843927e+02
max 7.612250e+03 7.700000e+03 7.570100e+03 7.625570e+03 7.625570e+03
Volume
count 1.569753e+06
mean 5.366149e+06
std 1.306298e+07
min 0.000000e+00
25% 9.724060e+05
50% 2.034200e+06
75% 4.665700e+06
max 9.140820e+08
# Check for null values in the DataFrame
null_values = combined_data.isnull().sum()
print("Count of Null Values:")
print(null_values)
if null_values.sum() == 0:
print("\nNo null values found in the DataFrame.")
else:
print("\nThere are null values in the DataFrame.")
Count of Null Values: Open 0 High 0 Low 0 Close 0 Adj Close 0 Volume 0 Ticker 0 date 0 dtype: int64 No null values found in the DataFrame.
# Check for data type in the DataFrame
num_rows, num_cols = null_values = combined_data.shape
print("\nNumber of Rows:", num_rows)
print("Number of Columns:", num_cols)
combined_data.dtypes
Number of Rows: 1569753 Number of Columns: 8
Open float64 High float64 Low float64 Close float64 Adj Close float64 Volume int64 Ticker object date datetime64[ns] dtype: object
# Find top 4 stocks based on mean market capitalization
ticker_market_caps = combined_data.groupby('Ticker')['Volume'].mean()
sorted_tickers = ticker_market_caps.sort_values(ascending=False)
stock_list = sorted_tickers.head(4)
print("Top 4 Stocks:")
print(stock_list)
Top 4 Stocks: Ticker TSLA 1.321701e+08 AAPL 9.732060e+07 AMZN 7.433595e+07 F 7.101031e+07 Name: Volume, dtype: float64
# Top_4_stocks based on trading volume
stock_list = stock_list.index.tolist()
print("Top 4 Tickers:")
print(stock_list)
Top 4 Tickers: ['TSLA', 'AAPL', 'AMZN', 'F']
# Create subplots for multiple visualizations
fig, axes = plt.subplots(nrows=2, ncols=2, figsize=(12, 8))
combined_data[['Open', 'High', 'Low', 'Close']].plot(ax=axes[0, 0])
axes[0, 0].set_title('Open, High, Low, Close Prices Over Time')
# Plot Volume over time
combined_data['Volume'].plot(ax=axes[0, 1])
axes[0, 1].set_title('Volume Over Time')
# Boxplot for Open, High, Low, and Close Prices
sns.boxplot(data=combined_data[['Open', 'High', 'Low', 'Close']], ax=axes[1, 0])
axes[1, 0].set_title('Boxplot for Open, High, Low, Close Prices')
# Boxplot for Volume
sns.boxplot(data=combined_data['Volume'], ax=axes[1, 1])
axes[1, 1].set_title('Boxplot for Volume')
# Adjust layout for better visibility
plt.tight_layout()
# Show the plots
plt.show()
# Compute the correlation matrix
corr_matrix = combined_data[['Open', 'High', 'Low', 'Close', 'Volume']].corr()
sns.heatmap(corr_matrix, annot=True, cmap='coolwarm', linewidths=0.5)
plt.title('Correlation Heatmap')
plt.show()
# Create a histogram for 'close' prices
plt.hist(combined_data['Close'], bins=30, color='blue', alpha=0.7)
plt.title('Histogram of Close Prices')
plt.xlabel('Close Price')
plt.ylabel('Frequency')
plt.show()
# Create a density plot for 'close' prices
sns.kdeplot(combined_data['Close'], fill=True, color='green')
plt.title('Density Plot of Close Prices')
plt.xlabel('Close Price')
plt.ylabel('Density')
plt.show()
fast_eda(combined_data)
DataFrame Head:
| Open | High | Low | Close | Adj Close | Volume | Ticker | date | |
|---|---|---|---|---|---|---|---|---|
| Date | ||||||||
| 2020-01-02 | 177.679993 | 180.009995 | 177.139999 | 180.000000 | 149.672821 | 3601700 | MMM | 2020-01-02 |
| 2020-01-03 | 177.020004 | 178.660004 | 175.630005 | 178.449997 | 148.383957 | 2466900 | MMM | 2020-01-03 |
| 2020-01-06 | 177.149994 | 178.710007 | 176.350006 | 178.619995 | 148.525299 | 1998000 | MMM | 2020-01-06 |
DataFrame Tail:
| Open | High | Low | Close | Adj Close | Volume | Ticker | date | |
|---|---|---|---|---|---|---|---|---|
| Date | ||||||||
| 2024-02-27 | 197.649994 | 199.250000 | 194.039993 | 199.000000 | 199.000000 | 1982500 | combined | 2024-02-27 |
| 2024-02-28 | 199.300003 | 200.500000 | 197.100006 | 199.940002 | 199.940002 | 2003300 | combined | 2024-02-28 |
| 2024-02-29 | 200.500000 | 200.529999 | 198.190002 | 198.330002 | 198.330002 | 2849200 | combined | 2024-02-29 |
----------------------------------------------------------------------------------------------------
Missing values:
| 0 |
|---|
---------------------------------------------------------------------------------------------------- Shape of DataFrame: (1046502, 8) ---------------------------------------------------------------------------------------------------- DataFrame Info: <class 'pandas.core.frame.DataFrame'> DatetimeIndex: 1046502 entries, 2020-01-02 to 2024-02-29 Data columns (total 8 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Open 1046502 non-null float64 1 High 1046502 non-null float64 2 Low 1046502 non-null float64 3 Close 1046502 non-null float64 4 Adj Close 1046502 non-null float64 5 Volume 1046502 non-null int64 6 Ticker 1046502 non-null object 7 date 1046502 non-null datetime64[ns] dtypes: datetime64[ns](1), float64(5), int64(1), object(1) memory usage: 71.9+ MB ---------------------------------------------------------------------------------------------------- Describe DataFrame:
| count | mean | median | std | min | 25% | 50% | 75% | max | skewness | kurtosis | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Open | 1046502 | 164.675 | 100.85 | 292.646 | 3.22 | 55.43 | 100.85 | 190.36 | 7612.25 | 11.235268 | 179.581375 |
| High | 1046502 | 166.758 | 102.21 | 296.327 | 3.29 | 56.17 | 102.21 | 192.75 | 7700 | 11.230501 | 179.236282 |
| Low | 1046502 | 162.581 | 99.51 | 289.135 | 3.02 | 54.68 | 99.51 | 187.98 | 7570.1 | 11.251279 | 180.274746 |
| Close | 1046502 | 164.707 | 100.87 | 292.804 | 3.12 | 55.42 | 100.87 | 190.43 | 7625.57 | 11.241679 | 179.816582 |
| Adj Close | 1046502 | 160.546 | 96.599 | 292.334 | 2.958 | 52.189 | 96.599 | 184.393 | 7625.57 | 11.318520 | 181.535487 |
| Volume | 1046502 | 5366148.547 | 2034200 | 13062978.959 | 0 | 972410.25 | 2034200 | 4665675 | 914082000 | 11.007764 | 258.012613 |
----------------------------------------------------------------------------------------------------
DataFrame Correlation:
/usr/local/lib/python3.10/dist-packages/fasteda/__init__.py:124: FutureWarning: The default value of numeric_only in DataFrame.corr is deprecated. In a future version, it will default to False. Select only valid columns or specify the value of numeric_only to silence this warning. sns.heatmap(df.corr(), annot = True, cmap = "Spectral", linewidths = 2, linecolor = "#000000", fmt='.3f')
----------------------------------------------------------------------------------------------------
DataFrame Pairplot:
----------------------------------------------------------------------------------------------------
Histogram(s) & Boxplot(s):
# Time Series of sortlisted stocks for Close
plt.figure(figsize=(20,12))
for i, company in enumerate(stock_list,1):
plt.subplot(2, 2, i)
df=combined_data[combined_data['Ticker']==company]
plt.plot(df['date'],df['Close'])
plt.title(company)
# Time Series of sortlisted stocks for Adj Close
plt.figure(figsize=(20,12))
for i, company in enumerate(stock_list,1):
plt.subplot(2, 2, i)
df=combined_data[combined_data['Ticker']==company]
plt.plot(df['date'],df['Adj Close'])
plt.title(company)
## Time Series of sortlisted stocks for Volume
plt.figure(figsize=(20,12))
for i, company in enumerate(stock_list,1):
plt.subplot(2, 2, i)
df=combined_data[combined_data['Ticker']==company]
plt.plot(df['date'],df['Volume'])
plt.title(company)
# Time Series of sortlisted stocks for Volume using Ploty dark
for company in (stock_list):
df=combined_data[combined_data['Ticker']==company]
fig = px.line(df, x="date", y="Volume", title=company,template = 'plotly_dark')
fig.show()
# Filter combined data for selected stocks
tesla_data = combined_data[combined_data['Ticker'] == 'TSLA']
apple_data = combined_data[combined_data['Ticker'] == 'AAPL']
amazon_data = combined_data[combined_data['Ticker'] == 'AMZN']
ford_data = combined_data[combined_data['Ticker'] == 'F']
print("Tesla Data:")
print(tesla_data.head())
Tesla Data:
Open High Low Close Adj Close Volume \
Date
2020-01-02 28.299999 28.713333 28.114000 28.684000 28.684000 142981500
2020-01-03 29.366667 30.266666 29.128000 29.534000 29.534000 266677500
2020-01-06 29.364668 30.104000 29.333332 30.102667 30.102667 151995000
2020-01-07 30.760000 31.441999 30.224001 31.270666 31.270666 268231500
2020-01-08 31.580000 33.232666 31.215334 32.809334 32.809334 467164500
Ticker date
Date
2020-01-02 TSLA 2020-01-02
2020-01-03 TSLA 2020-01-03
2020-01-06 TSLA 2020-01-06
2020-01-07 TSLA 2020-01-07
2020-01-08 TSLA 2020-01-08
close=pd.DataFrame()
close['Tesla']=tesla_data['Close']
close['Apple']=apple_data['Close']
close['Amazon']=amazon_data['Close']
close['Ford']=ford_data['Close']
close.head()
| Tesla | Apple | Amazon | Ford | |
|---|---|---|---|---|
| Date | ||||
| 2020-01-02 | 28.684000 | 75.087502 | 94.900497 | 9.42 |
| 2020-01-03 | 29.534000 | 74.357498 | 93.748497 | 9.21 |
| 2020-01-06 | 30.102667 | 74.949997 | 95.143997 | 9.16 |
| 2020-01-07 | 31.270666 | 74.597504 | 95.343002 | 9.25 |
| 2020-01-08 | 32.809334 | 75.797501 | 94.598503 | 9.25 |
#sns.pairplot(data=close)
sns.pairplot(close, kind='reg')
<seaborn.axisgrid.PairGrid at 0x7bd7c1902290>
sns.heatmap(close.corr(),annot=True,cmap='Blues',linecolor="black")
<Axes: >
Above we can see all the relationships on daily returns between all 4 stocks. A quick glance shows an interesting correlation between Ford and Tesla. It might be interesting to investigate that individual comaprison.
# Calculate the 90-day moving average for each stock
moving_avg_90 = close.rolling(window=90).mean()
# Calculate the 180-day moving average for each stock
moving_avg_180 = close.rolling(window=180).mean()
# Display the moving averages
print("90-Day Moving Average:")
print(moving_avg_90.tail()) # Display the last few rows
print("\n180-Day Moving Average:")
print(moving_avg_180.tail()) # Display the last few rows
90-Day Moving Average:
Tesla Apple Amazon Ford
Date
2024-02-23 222.086556 186.390667 149.913666 11.343222
2024-02-26 221.480778 186.417778 150.382333 11.343445
2024-02-27 220.868333 186.478667 150.849777 11.343000
2024-02-28 220.416778 186.540667 151.350111 11.349111
2024-02-29 220.214222 186.599444 151.887444 11.359222
180-Day Moving Average:
Tesla Apple Amazon Ford
Date
2024-02-23 239.191944 184.708000 140.976611 12.228944
2024-02-26 239.052111 184.726556 141.273833 12.219833
2024-02-27 238.856944 184.738000 141.547667 12.211056
2024-02-28 238.621611 184.740556 141.823945 12.203056
2024-02-29 238.355222 184.723667 142.102778 12.195611
# Plotting
plt.figure(figsize=(20, 12))
for stock in close.columns:
plt.plot(close.index, close[stock], label=f'{stock} Close Price')
plt.plot(close.index, moving_avg_90[stock], label=f'{stock} 90-Day Moving Average', linestyle='--')
plt.plot(close.index, moving_avg_180[stock], label=f'{stock} 180-Day Moving Average', linestyle=':')
plt.title('Close Price and Moving Averages for All Stocks')
plt.xlabel('Date')
plt.ylabel('Price')
plt.legend()
plt.xticks(rotation=45)
plt.grid(True)
plt.show()
# Determine the number of subplots needed
num_plots = len(close.columns)
num_rows = 2
num_cols = 2
fig, axs = plt.subplots(num_rows, num_cols, figsize=(25, 12))
axs = axs.flatten()
for i, stock in enumerate(close.columns):
axs[i].plot(close.index, close[stock], label='Close Price', color='blue')
axs[i].plot(close.index, moving_avg_90[stock], label='90-Day Moving Average', color='red')
axs[i].plot(close.index, moving_avg_180[stock], label='180-Day Moving Average', color='green')
axs[i].set_title(f'{stock} Close Price and Moving Averages')
axs[i].set_xlabel('Date')
axs[i].set_ylabel('Price')
axs[i].legend()
axs[i].tick_params(axis='x', rotation=45)
axs[i].grid(True)
plt.tight_layout()
plt.show()
# Perform time series decomposition for each stock
decomposition = {}
for stock in close.columns:
result = seasonal_decompose(close[stock], model='additive', period=365)
decomposition[stock] = result
for stock, result in decomposition.items():
plt.figure(figsize=(20, 20))
plt.subplot(411)
plt.plot(result.observed, label='Original', color='blue')
plt.legend(loc='upper left')
plt.subplot(412)
plt.plot(result.trend, label='Trend', color='red')
plt.legend(loc='upper left')
plt.subplot(413)
plt.plot(result.seasonal, label='Seasonal', color='green')
plt.legend(loc='upper left')
plt.subplot(414)
plt.plot(result.resid, label='Residual', color='purple')
plt.legend(loc='upper left')
plt.suptitle(f'Decomposition of {stock} Close Price')
plt.tight_layout()
plt.show()
# Iterate over each stock and perform the ADF test
for stock in close.columns:
ts = close[stock]
result = adfuller(ts)
adf_statistic = result[0]
p_value = result[1]
critical_values = result[4]
print(f'Stock: {stock}')
print(f'ADF Statistic: {adf_statistic}')
print(f'p-value: {p_value}')
print('Critical Values:')
for key, value in critical_values.items():
print(f' {key}: {value}')
if p_value < 0.05:
print('Result: Series is likely stationary (reject null hypothesis)')
else:
print('Result: Series is likely non-stationary (fail to reject null hypothesis)')
print()
Stock: Tesla ADF Statistic: -2.508930972626126 p-value: 0.1133164174563997 Critical Values: 1%: -3.436671659540904 5%: -2.8643311477003515 10%: -2.5682561222519897 Result: Series is likely non-stationary (fail to reject null hypothesis) Stock: Apple ADF Statistic: -1.7984539561652384 p-value: 0.38122953117045855 Critical Values: 1%: -3.436617132314828 5%: -2.864307097893787 10%: -2.568243313067353 Result: Series is likely non-stationary (fail to reject null hypothesis) Stock: Amazon ADF Statistic: -1.8994410397299941 p-value: 0.3323241089508116 Critical Values: 1%: -3.436617132314828 5%: -2.864307097893787 10%: -2.568243313067353 Result: Series is likely non-stationary (fail to reject null hypothesis) Stock: Ford ADF Statistic: -1.9927383413967492 p-value: 0.28976613221893677 Critical Values: 1%: -3.4367333690404767 5%: -2.8643583648001925 10%: -2.568270618452702 Result: Series is likely non-stationary (fail to reject null hypothesis)
# Iterate over each stock and check for trend and seasonality using autocorrelation
for stock in close.columns:
ts = close[stock]
autocorr = acf(ts, fft=False)
trend_detected = autocorr[1] > 0.5
seasonal_detected = any(autocorr[lag] > 0.5 for lag in range(7, 31))
print(f'Stock: {stock}')
if trend_detected:
print('Trend detected')
else:
print('No trend detected')
if seasonal_detected:
print('Seasonality detected')
else:
print('No seasonality detected')
print()
Stock: Tesla Trend detected Seasonality detected Stock: Apple Trend detected Seasonality detected Stock: Amazon Trend detected Seasonality detected Stock: Ford Trend detected Seasonality detected
# Function to perform the ADF test and check stationarity
def check_stationarity(data, stock_name, transformation_name):
result = adfuller(data)
p_value = result[1]
if p_value < 0.05:
print(f"{stock_name} ({transformation_name}): ADF Statistic:", result[0])
print(f"{stock_name} ({transformation_name}): p-value:", p_value)
print(f"{stock_name} ({transformation_name}): Reject the null hypothesis (data is stationary)")
else:
print(f"{stock_name} ({transformation_name}): ADF Statistic:", result[0])
print(f"{stock_name} ({transformation_name}): p-value:", p_value)
print(f"{stock_name} ({transformation_name}): Fail to reject the null hypothesis (data is non-stationary)")
for stock_name in close.columns:
print("\nStock:", stock_name)
ts = close[stock_name]
print("Original Data:")
check_stationarity(ts, stock_name, "Original")
# First-order differencing
print("\nFirst-order Differencing:")
ts_diff = ts.diff().dropna()
check_stationarity(ts_diff, stock_name, "First-order Differencing")
# Logarithmic transformation
print("\nLogarithmic Transformation:")
ts_log = np.log(ts)
check_stationarity(ts_log, stock_name, "Logarithmic Transformation")
# Box-Cox Transformation
print("\nBox-Cox Transformation:")
ts_boxcox, _ = boxcox(ts)
check_stationarity(ts_boxcox, stock_name, "Box-Cox Transformation")
Stock: Tesla Original Data: Tesla (Original): ADF Statistic: -2.508930972626126 Tesla (Original): p-value: 0.1133164174563997 Tesla (Original): Fail to reject the null hypothesis (data is non-stationary) First-order Differencing: Tesla (First-order Differencing): ADF Statistic: -9.332542763008549 Tesla (First-order Differencing): p-value: 9.245353038076098e-16 Tesla (First-order Differencing): Reject the null hypothesis (data is stationary) Logarithmic Transformation: Tesla (Logarithmic Transformation): ADF Statistic: -3.319207332842492 Tesla (Logarithmic Transformation): p-value: 0.01404143513690903 Tesla (Logarithmic Transformation): Reject the null hypothesis (data is stationary) Box-Cox Transformation: Tesla (Box-Cox Transformation): ADF Statistic: -2.5143436530272614 Tesla (Box-Cox Transformation): p-value: 0.11202325085143111 Tesla (Box-Cox Transformation): Fail to reject the null hypothesis (data is non-stationary) Stock: Apple Original Data: Apple (Original): ADF Statistic: -1.7984539561652384 Apple (Original): p-value: 0.38122953117045855 Apple (Original): Fail to reject the null hypothesis (data is non-stationary) First-order Differencing: Apple (First-order Differencing): ADF Statistic: -33.39772071161903 Apple (First-order Differencing): p-value: 0.0 Apple (First-order Differencing): Reject the null hypothesis (data is stationary) Logarithmic Transformation: Apple (Logarithmic Transformation): ADF Statistic: -1.914122732950772 Apple (Logarithmic Transformation): p-value: 0.32544677573603575 Apple (Logarithmic Transformation): Fail to reject the null hypothesis (data is non-stationary) Box-Cox Transformation: Apple (Box-Cox Transformation): ADF Statistic: -1.6634591706048383 Apple (Box-Cox Transformation): p-value: 0.4500308740323451 Apple (Box-Cox Transformation): Fail to reject the null hypothesis (data is non-stationary) Stock: Amazon Original Data: Amazon (Original): ADF Statistic: -1.8994410397299941 Amazon (Original): p-value: 0.3323241089508116 Amazon (Original): Fail to reject the null hypothesis (data is non-stationary) First-order Differencing: Amazon (First-order Differencing): ADF Statistic: -32.991375934978585 Amazon (First-order Differencing): p-value: 0.0 Amazon (First-order Differencing): Reject the null hypothesis (data is stationary) Logarithmic Transformation: Amazon (Logarithmic Transformation): ADF Statistic: -1.9734711451741194 Amazon (Logarithmic Transformation): p-value: 0.2983248224454217 Amazon (Logarithmic Transformation): Fail to reject the null hypothesis (data is non-stationary) Box-Cox Transformation: Amazon (Box-Cox Transformation): ADF Statistic: -1.891093228427692 Amazon (Box-Cox Transformation): p-value: 0.336262668290978 Amazon (Box-Cox Transformation): Fail to reject the null hypothesis (data is non-stationary) Stock: Ford Original Data: Ford (Original): ADF Statistic: -1.9927383413967492 Ford (Original): p-value: 0.28976613221893677 Ford (Original): Fail to reject the null hypothesis (data is non-stationary) First-order Differencing: Ford (First-order Differencing): ADF Statistic: -31.107163119871892 Ford (First-order Differencing): p-value: 0.0 Ford (First-order Differencing): Reject the null hypothesis (data is stationary) Logarithmic Transformation: Ford (Logarithmic Transformation): ADF Statistic: -1.6174802251765108 Ford (Logarithmic Transformation): p-value: 0.47405056419435393 Ford (Logarithmic Transformation): Fail to reject the null hypothesis (data is non-stationary) Box-Cox Transformation: Ford (Box-Cox Transformation): ADF Statistic: -1.522474746257365 Ford (Box-Cox Transformation): p-value: 0.5223655614334552 Ford (Box-Cox Transformation): Fail to reject the null hypothesis (data is non-stationary)
# Function to visualize the data
def plot_data(data, title):
plt.figure(figsize=(10, 6))
plt.plot(data.index, data.values, label='Original Data')
plt.xlabel('Date')
plt.ylabel('Value')
plt.title(title)
plt.legend()
plt.show()
# Function to perform ACF and PACF analysis
def acf_pacf_analysis(data, title):
plt.figure(figsize=(12, 6))
plot_acf(data, lags=20)
plt.title(f'Autocorrelation Function (ACF) - {title}')
plt.show()
plt.figure(figsize=(12, 6))
plot_pacf(data, lags=20)
plt.title(f'Partial Autocorrelation Function (PACF) - {title}')
plt.show()
# Function for ARIMA model
def fit_arima(data, order):
model_arima = ARIMA(data, order=order)
model_fit_arima = model_arima.fit()
return model_fit_arima
# Function for SARIMA model
def fit_sarima(data, order, seasonal_order):
model_sarima = SARIMAX(data, order=order, seasonal_order=seasonal_order)
model_fit_sarima = model_sarima.fit()
return model_fit_sarima
# Function for Exponential Smoothing model
def fit_exponential_smoothing(data):
model_es = ExponentialSmoothing(data)
model_fit_es = model_es.fit()
return model_fit_es
# Iterate over each stock and perform model selection
for stock_name in close.columns:
print("\nStock:", stock_name)
ts_diff = close[stock_name].diff().dropna()
plot_data(ts_diff, f'Transformed Data - {stock_name}')
acf_pacf_analysis(ts_diff, stock_name)
# Fit ARIMA model
model_fit_arima = fit_arima(ts_diff, order=(1, 1, 1))
print("ARIMA Model Summary:")
print(model_fit_arima.summary())
# Fit SARIMA model
model_fit_sarima = fit_sarima(ts_diff, order=(1, 1, 1), seasonal_order=(1, 1, 1, 12))
print("SARIMA Model Summary:")
print(model_fit_sarima.summary())
# Fit Exponential Smoothing model
model_fit_es = fit_exponential_smoothing(ts_diff)
print("Exponential Smoothing Model Summary:")
print(model_fit_es.summary())
Stock: Tesla
<Figure size 1200x600 with 0 Axes>
<Figure size 1200x600 with 0 Axes>
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting. /usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting. /usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.
ARIMA Model Summary:
SARIMAX Results
==============================================================================
Dep. Variable: Tesla No. Observations: 1046
Model: ARIMA(1, 1, 1) Log Likelihood -3717.950
Date: Fri, 29 Mar 2024 AIC 7441.900
Time: 12:17:45 BIC 7456.755
Sample: 0 HQIC 7447.534
- 1046
Covariance Type: opg
==============================================================================
coef std err z P>|z| [0.025 0.975]
------------------------------------------------------------------------------
ar.L1 -0.0287 0.027 -1.064 0.287 -0.082 0.024
ma.L1 -0.9986 0.005 -186.835 0.000 -1.009 -0.988
sigma2 71.6794 1.858 38.588 0.000 68.039 75.320
===================================================================================
Ljung-Box (L1) (Q): 0.00 Jarque-Bera (JB): 633.80
Prob(Q): 0.96 Prob(JB): 0.00
Heteroskedasticity (H): 1.09 Skew: -0.18
Prob(H) (two-sided): 0.43 Kurtosis: 6.80
===================================================================================
Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting. /usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.
SARIMA Model Summary:
SARIMAX Results
==========================================================================================
Dep. Variable: Tesla No. Observations: 1046
Model: SARIMAX(1, 1, 1)x(1, 1, 1, 12) Log Likelihood -3705.625
Date: Fri, 29 Mar 2024 AIC 7421.250
Time: 12:17:59 BIC 7445.951
Sample: 0 HQIC 7430.623
- 1046
Covariance Type: opg
==============================================================================
coef std err z P>|z| [0.025 0.975]
------------------------------------------------------------------------------
ar.L1 -0.0269 0.028 -0.970 0.332 -0.081 0.027
ma.L1 -0.9999 0.339 -2.952 0.003 -1.664 -0.336
ar.S.L12 0.0223 0.032 0.690 0.490 -0.041 0.086
ma.S.L12 -0.9999 2.487 -0.402 0.688 -5.874 3.874
sigma2 71.6939 179.443 0.400 0.689 -280.008 423.395
===================================================================================
Ljung-Box (L1) (Q): 0.00 Jarque-Bera (JB): 603.44
Prob(Q): 0.99 Prob(JB): 0.00
Heteroskedasticity (H): 1.09 Skew: -0.26
Prob(H) (two-sided): 0.43 Kurtosis: 6.71
===================================================================================
Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).
Exponential Smoothing Model Summary:
ExponentialSmoothing Model Results
================================================================================
Dep. Variable: Tesla No. Observations: 1046
Model: ExponentialSmoothing SSE 75124.211
Optimized: True AIC 4474.781
Trend: None BIC 4484.687
Seasonal: None AICC 4474.820
Seasonal Periods: None Date: Fri, 29 Mar 2024
Box-Cox: False Time: 12:17:59
Box-Cox Coeff.: None
==============================================================================
coeff code optimized
------------------------------------------------------------------------------
smoothing_level 0.0050151 alpha True
initial_level 0.5013415 l.0 True
------------------------------------------------------------------------------
Stock: Apple
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.
<Figure size 1200x600 with 0 Axes>
<Figure size 1200x600 with 0 Axes>
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting. /usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting. /usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.
ARIMA Model Summary:
SARIMAX Results
==============================================================================
Dep. Variable: Apple No. Observations: 1046
Model: ARIMA(1, 1, 1) Log Likelihood -2497.041
Date: Fri, 29 Mar 2024 AIC 5000.081
Time: 12:18:01 BIC 5014.937
Sample: 0 HQIC 5005.715
- 1046
Covariance Type: opg
==============================================================================
coef std err z P>|z| [0.025 0.975]
------------------------------------------------------------------------------
ar.L1 -0.0326 0.027 -1.226 0.220 -0.085 0.019
ma.L1 -1.0000 0.252 -3.972 0.000 -1.493 -0.507
sigma2 6.9202 1.752 3.950 0.000 3.486 10.354
===================================================================================
Ljung-Box (L1) (Q): 0.00 Jarque-Bera (JB): 120.09
Prob(Q): 0.96 Prob(JB): 0.00
Heteroskedasticity (H): 0.93 Skew: -0.06
Prob(H) (two-sided): 0.53 Kurtosis: 4.66
===================================================================================
Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting. /usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.
SARIMA Model Summary:
SARIMAX Results
==========================================================================================
Dep. Variable: Apple No. Observations: 1046
Model: SARIMAX(1, 1, 1)x(1, 1, 1, 12) Log Likelihood -2499.677
Date: Fri, 29 Mar 2024 AIC 5009.353
Time: 12:18:12 BIC 5034.054
Sample: 0 HQIC 5018.726
- 1046
Covariance Type: opg
==============================================================================
coef std err z P>|z| [0.025 0.975]
------------------------------------------------------------------------------
ar.L1 -0.0314 0.028 -1.140 0.254 -0.085 0.023
ma.L1 -0.9996 0.077 -13.021 0.000 -1.150 -0.849
ar.S.L12 -0.0259 0.031 -0.844 0.399 -0.086 0.034
ma.S.L12 -0.9995 0.625 -1.598 0.110 -2.225 0.226
sigma2 6.9389 4.357 1.593 0.111 -1.600 15.478
===================================================================================
Ljung-Box (L1) (Q): 0.00 Jarque-Bera (JB): 106.67
Prob(Q): 0.96 Prob(JB): 0.00
Heteroskedasticity (H): 0.94 Skew: -0.02
Prob(H) (two-sided): 0.57 Kurtosis: 4.57
===================================================================================
Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).
Exponential Smoothing Model Summary:
ExponentialSmoothing Model Results
================================================================================
Dep. Variable: Apple No. Observations: 1046
Model: ExponentialSmoothing SSE 7239.659
Optimized: True AIC 2027.592
Trend: None BIC 2037.498
Seasonal: None AICC 2027.631
Seasonal Periods: None Date: Fri, 29 Mar 2024
Box-Cox: False Time: 12:18:12
Box-Cox Coeff.: None
==============================================================================
coeff code optimized
------------------------------------------------------------------------------
smoothing_level 1.4901e-08 alpha True
initial_level 0.1010158 l.0 True
------------------------------------------------------------------------------
Stock: Amazon
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.
<Figure size 1200x600 with 0 Axes>
<Figure size 1200x600 with 0 Axes>
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting. /usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting. /usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.
ARIMA Model Summary:
SARIMAX Results
==============================================================================
Dep. Variable: Amazon No. Observations: 1046
Model: ARIMA(1, 1, 1) Log Likelihood -2667.438
Date: Fri, 29 Mar 2024 AIC 5340.875
Time: 12:18:14 BIC 5355.730
Sample: 0 HQIC 5346.509
- 1046
Covariance Type: opg
==============================================================================
coef std err z P>|z| [0.025 0.975]
------------------------------------------------------------------------------
ar.L1 -0.0209 0.026 -0.797 0.426 -0.072 0.031
ma.L1 -0.9999 0.051 -19.718 0.000 -1.099 -0.901
sigma2 9.5893 0.520 18.435 0.000 8.570 10.609
===================================================================================
Ljung-Box (L1) (Q): 0.00 Jarque-Bera (JB): 707.24
Prob(Q): 0.97 Prob(JB): 0.00
Heteroskedasticity (H): 0.77 Skew: -0.07
Prob(H) (two-sided): 0.01 Kurtosis: 7.03
===================================================================================
Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting. /usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.
SARIMA Model Summary:
SARIMAX Results
==========================================================================================
Dep. Variable: Amazon No. Observations: 1046
Model: SARIMAX(1, 1, 1)x(1, 1, 1, 12) Log Likelihood -2667.867
Date: Fri, 29 Mar 2024 AIC 5345.733
Time: 12:18:23 BIC 5370.434
Sample: 0 HQIC 5355.107
- 1046
Covariance Type: opg
==============================================================================
coef std err z P>|z| [0.025 0.975]
------------------------------------------------------------------------------
ar.L1 -0.0213 0.028 -0.759 0.448 -0.076 0.034
ma.L1 -0.9938 0.007 -141.399 0.000 -1.008 -0.980
ar.S.L12 -0.0180 0.032 -0.570 0.568 -0.080 0.044
ma.S.L12 -0.9946 0.065 -15.187 0.000 -1.123 -0.866
sigma2 9.6900 0.601 16.115 0.000 8.512 10.869
===================================================================================
Ljung-Box (L1) (Q): 0.00 Jarque-Bera (JB): 661.93
Prob(Q): 0.96 Prob(JB): 0.00
Heteroskedasticity (H): 0.79 Skew: 0.01
Prob(H) (two-sided): 0.03 Kurtosis: 6.92
===================================================================================
Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).
Exponential Smoothing Model Summary:
ExponentialSmoothing Model Results
================================================================================
Dep. Variable: Amazon No. Observations: 1046
Model: ExponentialSmoothing SSE 10024.847
Optimized: True AIC 2368.058
Trend: None BIC 2377.963
Seasonal: None AICC 2368.096
Seasonal Periods: None Date: Fri, 29 Mar 2024
Box-Cox: False Time: 12:18:23
Box-Cox Coeff.: None
==============================================================================
coeff code optimized
------------------------------------------------------------------------------
smoothing_level 1.4901e-08 alpha True
initial_level 0.0782595 l.0 True
------------------------------------------------------------------------------
Stock: Ford
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.
<Figure size 1200x600 with 0 Axes>
<Figure size 1200x600 with 0 Axes>
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting. /usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting. /usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.
ARIMA Model Summary:
SARIMAX Results
==============================================================================
Dep. Variable: Ford No. Observations: 1046
Model: ARIMA(1, 1, 1) Log Likelihood -396.431
Date: Fri, 29 Mar 2024 AIC 798.863
Time: 12:18:26 BIC 813.718
Sample: 0 HQIC 804.497
- 1046
Covariance Type: opg
==============================================================================
coef std err z P>|z| [0.025 0.975]
------------------------------------------------------------------------------
ar.L1 0.0385 0.021 1.813 0.070 -0.003 0.080
ma.L1 -0.9995 0.015 -64.614 0.000 -1.030 -0.969
sigma2 0.1243 0.003 40.394 0.000 0.118 0.130
===================================================================================
Ljung-Box (L1) (Q): 0.00 Jarque-Bera (JB): 1580.57
Prob(Q): 0.99 Prob(JB): 0.00
Heteroskedasticity (H): 1.24 Skew: 0.03
Prob(H) (two-sided): 0.05 Kurtosis: 9.02
===================================================================================
Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting. /usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.
SARIMA Model Summary:
SARIMAX Results
==========================================================================================
Dep. Variable: Ford No. Observations: 1046
Model: SARIMAX(1, 1, 1)x(1, 1, 1, 12) Log Likelihood -422.987
Date: Fri, 29 Mar 2024 AIC 855.973
Time: 12:18:39 BIC 880.675
Sample: 0 HQIC 865.347
- 1046
Covariance Type: opg
==============================================================================
coef std err z P>|z| [0.025 0.975]
------------------------------------------------------------------------------
ar.L1 0.0355 0.022 1.643 0.100 -0.007 0.078
ma.L1 -0.9979 0.012 -85.196 0.000 -1.021 -0.975
ar.S.L12 0.0183 0.024 0.760 0.447 -0.029 0.065
ma.S.L12 -0.9993 0.501 -1.993 0.046 -1.982 -0.017
sigma2 0.1248 0.062 2.025 0.043 0.004 0.246
===================================================================================
Ljung-Box (L1) (Q): 0.01 Jarque-Bera (JB): 1398.64
Prob(Q): 0.94 Prob(JB): 0.00
Heteroskedasticity (H): 1.16 Skew: 0.07
Prob(H) (two-sided): 0.18 Kurtosis: 8.70
===================================================================================
Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).
Exponential Smoothing Model Summary:
ExponentialSmoothing Model Results
================================================================================
Dep. Variable: Ford No. Observations: 1046
Model: ExponentialSmoothing SSE 129.987
Optimized: True AIC -2177.220
Trend: None BIC -2167.315
Seasonal: None AICC -2177.182
Seasonal Periods: None Date: Fri, 29 Mar 2024
Box-Cox: False Time: 12:18:39
Box-Cox Coeff.: None
==============================================================================
coeff code optimized
------------------------------------------------------------------------------
smoothing_level 1.4901e-08 alpha True
initial_level 0.0028872 l.0 True
------------------------------------------------------------------------------
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.
Tesla
ARIMA Model:
AIC: 7441.900 BIC: 7456.755 Log Likelihood: -3717.950 Ljung-Box (L1) (Q): 0.00 Jarque-Bera (JB): 633.80 SARIMA Model:
AIC: 7421.250 BIC: 7445.951 Log Likelihood: -3705.625 Ljung-Box (L1) (Q): 0.00 Jarque-Bera (JB): 603.44 Exponential Smoothing Model:
SSE: 75124.211 AIC: 4474.781 BIC: 4484.687
Apple
ARIMA Model:
AIC: 5000.081 BIC: 5014.937 Log Likelihood: -2497.041 SARIMA Model:
AIC: 5009.353 BIC: 5034.054 Log Likelihood: -2499.677 Exponential Smoothing Model:
AIC: 2027.592 BIC: 2037.498 SSE: 7239.659
Amazon
ARIMA Model:
AIC: 5340.875 BIC: 5355.730 Log Likelihood: -2667.438 SARIMA Model:
AIC: 5345.733 BIC: 5370.434 Log Likelihood: -2667.867 Exponential Smoothing Model:
AIC: 2368.058 BIC: 2377.963 SSE: 10024.847
Ford
ARIMA Model:
AIC: 798.863 BIC: 813.718 Log Likelihood: -396.431 SARIMA Model:
AIC: 855.973 BIC: 880.675 Log Likelihood: -422.987 Exponential Smoothing Model:
AIC: -2177.220 BIC: -2167.315 SSE: 129.987
# Perform model selection
for stock_name in ['Tesla', 'Apple', 'Amazon', 'Ford']:
print("\nStock:", stock_name)
ts_diff = close[stock_name].diff().dropna()
aic_values = []
bic_values = []
# ARIMA Model
aic_values.append(798.863)
bic_values.append(813.718)
# SARIMA Model
aic_values.append(855.973)
bic_values.append(880.675)
# Exponential Smoothing Model
aic_values.append(-2177.220)
bic_values.append(-2167.315)
print(aic_values)
print(bic_values)
# Model Selection
best_model = None
min_aic = min(aic_values)
min_bic = min(bic_values)
if min_aic == min_bic:
best_model = "Exponential Smoothing"
elif min_aic < min_bic:
best_model = "ARIMA"
else:
best_model = "SARIMA"
print("Best Model:", best_model)
Stock: Tesla [798.863, 855.973, -2177.22] [813.718, 880.675, -2167.315] Best Model: ARIMA Stock: Apple [798.863, 855.973, -2177.22] [813.718, 880.675, -2167.315] Best Model: ARIMA Stock: Amazon [798.863, 855.973, -2177.22] [813.718, 880.675, -2167.315] Best Model: ARIMA Stock: Ford [798.863, 855.973, -2177.22] [813.718, 880.675, -2167.315] Best Model: ARIMA
# Split the data into a training and validation dataset
# Load the dataset
series = read_csv('/content/drive/MyDrive/data/tmsp2/AAPL_historical_data.csv', header=0, index_col=0, parse_dates=True, squeeze=True)
series = series['Close']
# Define the split point
split_point = len(series) - 100
# Split the dataset
dataset, validation = series[0:split_point], series[split_point:]
# Print the sizes of the dataset and validation sets
print('Dataset %d, Validation %d' % (len(dataset), len(validation)))
# Save the datasets to CSV files
dataset.to_csv('/content/drive/MyDrive/data/app_dataset.csv', header=False)
validation.to_csv('/content/drive/MyDrive/data/app_validation.csv', header=False)
<ipython-input-4-79f3e07e30f5>:5: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
series = read_csv('/content/drive/MyDrive/data/tmsp2/AAPL_historical_data.csv', header=0, index_col=0, parse_dates=True, squeeze=True)
Dataset 947, Validation 100
# Read the dataset and validation CSV files
dataset = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None)
validation = read_csv('/content/drive/MyDrive/data/app_validation.csv', header=None)
# Print the number of observations in each file
print('Number of observations in dataset:', len(dataset))
print('Number of observations in validation:', len(validation))
Number of observations in dataset: 947 Number of observations in validation: 100
dataset
| 0 | 1 | |
|---|---|---|
| 0 | 2020-01-02 | 75.087502 |
| 1 | 2020-01-03 | 74.357498 |
| 2 | 2020-01-06 | 74.949997 |
| 3 | 2020-01-07 | 74.597504 |
| 4 | 2020-01-08 | 75.797501 |
| ... | ... | ... |
| 942 | 2023-09-29 | 171.210007 |
| 943 | 2023-10-02 | 173.750000 |
| 944 | 2023-10-03 | 172.399994 |
| 945 | 2023-10-04 | 173.660004 |
| 946 | 2023-10-05 | 174.910004 |
947 rows × 2 columns
# load data
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True) # prepare data
X = series.values
X = X.astype('float32')
train_size = int(len(X) * 0.50)
train, test = X[0:train_size], X[train_size:]
# walk-forward validation
history = [x for x in train]
predictions = list()
for i in range(len(test)):
# predict
yhat = history[-1]
predictions.append(yhat)
# observation
obs = test[i]
history.append(obs)
print('>Predicted=%.3f, Expected=%.3f' % (yhat, obs))
# report performance
rmse = sqrt(mean_squared_error(test, predictions))
print('RMSE: %.3f' % rmse)
>Predicted=150.000, Expected=151.000 >Predicted=151.000, Expected=153.490 >Predicted=153.490, Expected=157.870 >Predicted=157.870, Expected=160.550 >Predicted=160.550, Expected=161.020 >Predicted=161.020, Expected=161.410 >Predicted=161.410, Expected=161.940 >Predicted=161.940, Expected=156.810 >Predicted=156.810, Expected=160.240 >Predicted=160.240, Expected=165.300 >Predicted=165.300, Expected=164.770 >Predicted=164.770, Expected=163.760 >Predicted=163.760, Expected=161.840 >Predicted=161.840, Expected=165.320 >Predicted=165.320, Expected=171.180 >Predicted=171.180, Expected=175.080 >Predicted=175.080, Expected=174.560 >Predicted=174.560, Expected=179.450 >Predicted=179.450, Expected=175.740 >Predicted=175.740, Expected=174.330 >Predicted=174.330, Expected=179.300 >Predicted=179.300, Expected=172.260 >Predicted=172.260, Expected=171.140 >Predicted=171.140, Expected=169.750 >Predicted=169.750, Expected=172.990 >Predicted=172.990, Expected=175.640 >Predicted=175.640, Expected=176.280 >Predicted=176.280, Expected=180.330 >Predicted=180.330, Expected=179.290 >Predicted=179.290, Expected=179.380 >Predicted=179.380, Expected=178.200 >Predicted=178.200, Expected=177.570 >Predicted=177.570, Expected=182.010 >Predicted=182.010, Expected=179.700 >Predicted=179.700, Expected=174.920 >Predicted=174.920, Expected=172.000 >Predicted=172.000, Expected=172.170 >Predicted=172.170, Expected=172.190 >Predicted=172.190, Expected=175.080 >Predicted=175.080, Expected=175.530 >Predicted=175.530, Expected=172.190 >Predicted=172.190, Expected=173.070 >Predicted=173.070, Expected=169.800 >Predicted=169.800, Expected=166.230 >Predicted=166.230, Expected=164.510 >Predicted=164.510, Expected=162.410 >Predicted=162.410, Expected=161.620 >Predicted=161.620, Expected=159.780 >Predicted=159.780, Expected=159.690 >Predicted=159.690, Expected=159.220 >Predicted=159.220, Expected=170.330 >Predicted=170.330, Expected=174.780 >Predicted=174.780, Expected=174.610 >Predicted=174.610, Expected=175.840 >Predicted=175.840, Expected=172.900 >Predicted=172.900, Expected=172.390 >Predicted=172.390, Expected=171.660 >Predicted=171.660, Expected=174.830 >Predicted=174.830, Expected=176.280 >Predicted=176.280, Expected=172.120 >Predicted=172.120, Expected=168.640 >Predicted=168.640, Expected=168.880 >Predicted=168.880, Expected=172.790 >Predicted=172.790, Expected=172.550 >Predicted=172.550, Expected=168.880 >Predicted=168.880, Expected=167.300 >Predicted=167.300, Expected=164.320 >Predicted=164.320, Expected=160.070 >Predicted=160.070, Expected=162.740 >Predicted=162.740, Expected=164.850 >Predicted=164.850, Expected=165.120 >Predicted=165.120, Expected=163.200 >Predicted=163.200, Expected=166.560 >Predicted=166.560, Expected=166.230 >Predicted=166.230, Expected=163.170 >Predicted=163.170, Expected=159.300 >Predicted=159.300, Expected=157.440 >Predicted=157.440, Expected=162.950 >Predicted=162.950, Expected=158.520 >Predicted=158.520, Expected=154.730 >Predicted=154.730, Expected=150.620 >Predicted=150.620, Expected=155.090 >Predicted=155.090, Expected=159.590 >Predicted=159.590, Expected=160.620 >Predicted=160.620, Expected=163.980 >Predicted=163.980, Expected=165.380 >Predicted=165.380, Expected=168.820 >Predicted=168.820, Expected=170.210 >Predicted=170.210, Expected=174.070 >Predicted=174.070, Expected=174.720 >Predicted=174.720, Expected=175.600 >Predicted=175.600, Expected=178.960 >Predicted=178.960, Expected=177.770 >Predicted=177.770, Expected=174.610 >Predicted=174.610, Expected=174.310 >Predicted=174.310, Expected=178.440 >Predicted=178.440, Expected=175.060 >Predicted=175.060, Expected=171.830 >Predicted=171.830, Expected=172.140 >Predicted=172.140, Expected=170.090 >Predicted=170.090, Expected=165.750 >Predicted=165.750, Expected=167.660 >Predicted=167.660, Expected=170.400 >Predicted=170.400, Expected=165.290 >Predicted=165.290, Expected=165.070 >Predicted=165.070, Expected=167.400 >Predicted=167.400, Expected=167.230 >Predicted=167.230, Expected=166.420 >Predicted=166.420, Expected=161.790 >Predicted=161.790, Expected=162.880 >Predicted=162.880, Expected=156.800 >Predicted=156.800, Expected=156.570 >Predicted=156.570, Expected=163.640 >Predicted=163.640, Expected=157.650 >Predicted=157.650, Expected=157.960 >Predicted=157.960, Expected=159.480 >Predicted=159.480, Expected=166.020 >Predicted=166.020, Expected=156.770 >Predicted=156.770, Expected=157.280 >Predicted=157.280, Expected=152.060 >Predicted=152.060, Expected=154.510 >Predicted=154.510, Expected=146.500 >Predicted=146.500, Expected=142.560 >Predicted=142.560, Expected=147.110 >Predicted=147.110, Expected=145.540 >Predicted=145.540, Expected=149.240 >Predicted=149.240, Expected=140.820 >Predicted=140.820, Expected=137.350 >Predicted=137.350, Expected=137.590 >Predicted=137.590, Expected=143.110 >Predicted=143.110, Expected=140.360 >Predicted=140.360, Expected=140.520 >Predicted=140.520, Expected=143.780 >Predicted=143.780, Expected=149.640 >Predicted=149.640, Expected=148.840 >Predicted=148.840, Expected=148.710 >Predicted=148.710, Expected=151.210 >Predicted=151.210, Expected=145.380 >Predicted=145.380, Expected=146.140 >Predicted=146.140, Expected=148.710 >Predicted=148.710, Expected=147.960 >Predicted=147.960, Expected=142.640 >Predicted=142.640, Expected=137.130 >Predicted=137.130, Expected=131.880 >Predicted=131.880, Expected=132.760 >Predicted=132.760, Expected=135.430 >Predicted=135.430, Expected=130.060 >Predicted=130.060, Expected=131.560 >Predicted=131.560, Expected=135.870 >Predicted=135.870, Expected=135.350 >Predicted=135.350, Expected=138.270 >Predicted=138.270, Expected=141.660 >Predicted=141.660, Expected=141.660 >Predicted=141.660, Expected=137.440 >Predicted=137.440, Expected=139.230 >Predicted=139.230, Expected=136.720 >Predicted=136.720, Expected=138.930 >Predicted=138.930, Expected=141.560 >Predicted=141.560, Expected=142.920 >Predicted=142.920, Expected=146.350 >Predicted=146.350, Expected=147.040 >Predicted=147.040, Expected=144.870 >Predicted=144.870, Expected=145.860 >Predicted=145.860, Expected=145.490 >Predicted=145.490, Expected=148.470 >Predicted=148.470, Expected=150.170 >Predicted=150.170, Expected=147.070 >Predicted=147.070, Expected=151.000 >Predicted=151.000, Expected=153.040 >Predicted=153.040, Expected=155.350 >Predicted=155.350, Expected=154.090 >Predicted=154.090, Expected=152.950 >Predicted=152.950, Expected=151.600 >Predicted=151.600, Expected=156.790 >Predicted=156.790, Expected=157.350 >Predicted=157.350, Expected=162.510 >Predicted=162.510, Expected=161.510 >Predicted=161.510, Expected=160.010 >Predicted=160.010, Expected=166.130 >Predicted=166.130, Expected=165.810 >Predicted=165.810, Expected=165.350 >Predicted=165.350, Expected=164.870 >Predicted=164.870, Expected=164.920 >Predicted=164.920, Expected=169.240 >Predicted=169.240, Expected=168.490 >Predicted=168.490, Expected=172.100 >Predicted=172.100, Expected=173.190 >Predicted=173.190, Expected=173.030 >Predicted=173.030, Expected=174.550 >Predicted=174.550, Expected=174.150 >Predicted=174.150, Expected=171.520 >Predicted=171.520, Expected=167.570 >Predicted=167.570, Expected=167.230 >Predicted=167.230, Expected=167.530 >Predicted=167.530, Expected=170.030 >Predicted=170.030, Expected=163.620 >Predicted=163.620, Expected=161.380 >Predicted=161.380, Expected=158.910 >Predicted=158.910, Expected=157.220 >Predicted=157.220, Expected=157.960 >Predicted=157.960, Expected=155.810 >Predicted=155.810, Expected=154.530 >Predicted=154.530, Expected=155.960 >Predicted=155.960, Expected=154.460 >Predicted=154.460, Expected=157.370 >Predicted=157.370, Expected=163.430 >Predicted=163.430, Expected=153.840 >Predicted=153.840, Expected=155.310 >Predicted=155.310, Expected=152.370 >Predicted=152.370, Expected=150.700 >Predicted=150.700, Expected=154.480 >Predicted=154.480, Expected=156.900 >Predicted=156.900, Expected=153.720 >Predicted=153.720, Expected=152.740 >Predicted=152.740, Expected=150.430 >Predicted=150.430, Expected=150.770 >Predicted=150.770, Expected=151.760 >Predicted=151.760, Expected=149.840 >Predicted=149.840, Expected=142.480 >Predicted=142.480, Expected=138.200 >Predicted=138.200, Expected=142.450 >Predicted=142.450, Expected=146.100 >Predicted=146.100, Expected=146.400 >Predicted=146.400, Expected=145.430 >Predicted=145.430, Expected=140.090 >Predicted=140.090, Expected=140.420 >Predicted=140.420, Expected=138.980 >Predicted=138.980, Expected=138.340 >Predicted=138.340, Expected=142.990 >Predicted=142.990, Expected=138.380 >Predicted=138.380, Expected=142.410 >Predicted=142.410, Expected=143.750 >Predicted=143.750, Expected=143.860 >Predicted=143.860, Expected=143.390 >Predicted=143.390, Expected=147.270 >Predicted=147.270, Expected=149.450 >Predicted=149.450, Expected=152.340 >Predicted=152.340, Expected=149.350 >Predicted=149.350, Expected=144.800 >Predicted=144.800, Expected=155.740 >Predicted=155.740, Expected=153.340 >Predicted=153.340, Expected=150.650 >Predicted=150.650, Expected=145.030 >Predicted=145.030, Expected=138.880 >Predicted=138.880, Expected=138.380 >Predicted=138.380, Expected=138.920 >Predicted=138.920, Expected=139.500 >Predicted=139.500, Expected=134.870 >Predicted=134.870, Expected=146.870 >Predicted=146.870, Expected=149.700 >Predicted=149.700, Expected=148.280 >Predicted=148.280, Expected=150.040 >Predicted=150.040, Expected=148.790 >Predicted=148.790, Expected=150.720 >Predicted=150.720, Expected=151.290 >Predicted=151.290, Expected=148.010 >Predicted=148.010, Expected=150.180 >Predicted=150.180, Expected=151.070 >Predicted=151.070, Expected=148.110 >Predicted=148.110, Expected=144.220 >Predicted=144.220, Expected=141.170 >Predicted=141.170, Expected=148.030 >Predicted=148.030, Expected=148.310 >Predicted=148.310, Expected=147.810 >Predicted=147.810, Expected=146.630 >Predicted=146.630, Expected=142.910 >Predicted=142.910, Expected=140.940 >Predicted=140.940, Expected=142.650 >Predicted=142.650, Expected=142.160 >Predicted=142.160, Expected=144.490 >Predicted=144.490, Expected=145.470 >Predicted=145.470, Expected=143.210 >Predicted=143.210, Expected=136.500 >Predicted=136.500, Expected=134.510 >Predicted=134.510, Expected=132.370 >Predicted=132.370, Expected=132.300 >Predicted=132.300, Expected=135.450 >Predicted=135.450, Expected=132.230 >Predicted=132.230, Expected=131.860 >Predicted=131.860, Expected=130.030 >Predicted=130.030, Expected=126.040 >Predicted=126.040, Expected=129.610 >Predicted=129.610, Expected=129.930 >Predicted=129.930, Expected=125.070 >Predicted=125.070, Expected=126.360 >Predicted=126.360, Expected=125.020 >Predicted=125.020, Expected=129.620 >Predicted=129.620, Expected=130.150 >Predicted=130.150, Expected=130.730 >Predicted=130.730, Expected=133.490 >Predicted=133.490, Expected=133.410 >Predicted=133.410, Expected=134.760 >Predicted=134.760, Expected=135.940 >Predicted=135.940, Expected=135.210 >Predicted=135.210, Expected=135.270 >Predicted=135.270, Expected=137.870 >Predicted=137.870, Expected=141.110 >Predicted=141.110, Expected=142.530 >Predicted=142.530, Expected=141.860 >Predicted=141.860, Expected=143.960 >Predicted=143.960, Expected=145.930 >Predicted=145.930, Expected=143.000 >Predicted=143.000, Expected=144.290 >Predicted=144.290, Expected=145.430 >Predicted=145.430, Expected=150.820 >Predicted=150.820, Expected=154.500 >Predicted=154.500, Expected=151.730 >Predicted=151.730, Expected=154.650 >Predicted=154.650, Expected=151.920 >Predicted=151.920, Expected=150.870 >Predicted=150.870, Expected=151.010 >Predicted=151.010, Expected=153.850 >Predicted=153.850, Expected=153.200 >Predicted=153.200, Expected=155.330 >Predicted=155.330, Expected=153.710 >Predicted=153.710, Expected=152.550 >Predicted=152.550, Expected=148.480 >Predicted=148.480, Expected=148.910 >Predicted=148.910, Expected=149.400 >Predicted=149.400, Expected=146.710 >Predicted=146.710, Expected=147.920 >Predicted=147.920, Expected=147.410 >Predicted=147.410, Expected=145.310 >Predicted=145.310, Expected=145.910 >Predicted=145.910, Expected=151.030 >Predicted=151.030, Expected=153.830 >Predicted=153.830, Expected=151.600 >Predicted=151.600, Expected=152.870 >Predicted=152.870, Expected=150.590 >Predicted=150.590, Expected=148.500 >Predicted=148.500, Expected=150.470 >Predicted=150.470, Expected=152.590 >Predicted=152.590, Expected=152.990 >Predicted=152.990, Expected=155.850 >Predicted=155.850, Expected=155.000 >Predicted=155.000, Expected=157.400 >Predicted=157.400, Expected=159.280 >Predicted=159.280, Expected=157.830 >Predicted=157.830, Expected=158.930 >Predicted=158.930, Expected=160.250 >Predicted=160.250, Expected=158.280 >Predicted=158.280, Expected=157.650 >Predicted=157.650, Expected=160.770 >Predicted=160.770, Expected=162.360 >Predicted=162.360, Expected=164.900 >Predicted=164.900, Expected=166.170 >Predicted=166.170, Expected=165.630 >Predicted=165.630, Expected=163.760 >Predicted=163.760, Expected=164.660 >Predicted=164.660, Expected=162.030 >Predicted=162.030, Expected=160.800 >Predicted=160.800, Expected=160.100 >Predicted=160.100, Expected=165.560 >Predicted=165.560, Expected=165.210 >Predicted=165.210, Expected=165.230 >Predicted=165.230, Expected=166.470 >Predicted=166.470, Expected=167.630 >Predicted=167.630, Expected=166.650 >Predicted=166.650, Expected=165.020 >Predicted=165.020, Expected=165.330 >Predicted=165.330, Expected=163.770 >Predicted=163.770, Expected=163.760 >Predicted=163.760, Expected=168.410 >Predicted=168.410, Expected=169.680 >Predicted=169.680, Expected=169.590 >Predicted=169.590, Expected=168.540 >Predicted=168.540, Expected=167.450 >Predicted=167.450, Expected=165.790 >Predicted=165.790, Expected=173.570 >Predicted=173.570, Expected=173.500 >Predicted=173.500, Expected=171.770 >Predicted=171.770, Expected=173.560 >Predicted=173.560, Expected=173.750 >Predicted=173.750, Expected=172.570 >Predicted=172.570, Expected=172.070 >Predicted=172.070, Expected=172.070 >Predicted=172.070, Expected=172.690 >Predicted=172.690, Expected=175.050 >Predicted=175.050, Expected=175.160 >Predicted=175.160, Expected=174.200 >Predicted=174.200, Expected=171.560 >Predicted=171.560, Expected=171.840 >Predicted=171.840, Expected=172.990 >Predicted=172.990, Expected=175.430 >Predicted=175.430, Expected=177.300 >Predicted=177.300, Expected=177.250 >Predicted=177.250, Expected=180.090 >Predicted=180.090, Expected=180.950 >Predicted=180.950, Expected=179.580 >Predicted=179.580, Expected=179.210 >Predicted=179.210, Expected=177.820 >Predicted=177.820, Expected=180.570 >Predicted=180.570, Expected=180.960 >Predicted=180.960, Expected=183.790 >Predicted=183.790, Expected=183.310 >Predicted=183.310, Expected=183.950 >Predicted=183.950, Expected=186.010 >Predicted=186.010, Expected=184.920 >Predicted=184.920, Expected=185.010 >Predicted=185.010, Expected=183.960 >Predicted=183.960, Expected=187.000 >Predicted=187.000, Expected=186.680 >Predicted=186.680, Expected=185.270 >Predicted=185.270, Expected=188.060 >Predicted=188.060, Expected=189.250 >Predicted=189.250, Expected=189.590 >Predicted=189.590, Expected=193.970 >Predicted=193.970, Expected=192.460 >Predicted=192.460, Expected=191.330 >Predicted=191.330, Expected=191.810 >Predicted=191.810, Expected=190.680 >Predicted=190.680, Expected=188.610 >Predicted=188.610, Expected=188.080 >Predicted=188.080, Expected=189.770 >Predicted=189.770, Expected=190.540 >Predicted=190.540, Expected=190.690 >Predicted=190.690, Expected=193.990 >Predicted=193.990, Expected=193.730 >Predicted=193.730, Expected=195.100 >Predicted=195.100, Expected=193.130 >Predicted=193.130, Expected=191.940 >Predicted=191.940, Expected=192.750 >Predicted=192.750, Expected=193.620 >Predicted=193.620, Expected=194.500 >Predicted=194.500, Expected=193.220 >Predicted=193.220, Expected=195.830 >Predicted=195.830, Expected=196.450 >Predicted=196.450, Expected=195.610 >Predicted=195.610, Expected=192.580 >Predicted=192.580, Expected=191.170 >Predicted=191.170, Expected=181.990 >Predicted=181.990, Expected=178.850 >Predicted=178.850, Expected=179.800 >Predicted=179.800, Expected=178.190 >Predicted=178.190, Expected=177.970 >Predicted=177.970, Expected=177.790 >Predicted=177.790, Expected=179.460 >Predicted=179.460, Expected=177.450 >Predicted=177.450, Expected=176.570 >Predicted=176.570, Expected=174.000 >Predicted=174.000, Expected=174.490 >Predicted=174.490, Expected=175.840 >Predicted=175.840, Expected=177.230 >Predicted=177.230, Expected=181.120 >Predicted=181.120, Expected=176.380 >Predicted=176.380, Expected=178.610 >Predicted=178.610, Expected=180.190 >Predicted=180.190, Expected=184.120 >Predicted=184.120, Expected=187.650 >Predicted=187.650, Expected=187.870 >Predicted=187.870, Expected=189.460 >Predicted=189.460, Expected=189.700 >Predicted=189.700, Expected=182.910 >Predicted=182.910, Expected=177.560 >Predicted=177.560, Expected=178.180 >Predicted=178.180, Expected=179.360 >Predicted=179.360, Expected=176.300 >Predicted=176.300, Expected=174.210 >Predicted=174.210, Expected=175.740 >Predicted=175.740, Expected=175.010 >Predicted=175.010, Expected=177.970 >Predicted=177.970, Expected=179.070 >Predicted=179.070, Expected=175.490 >Predicted=175.490, Expected=173.930 >Predicted=173.930, Expected=174.790 >Predicted=174.790, Expected=176.080 >Predicted=176.080, Expected=171.960 >Predicted=171.960, Expected=170.430 >Predicted=170.430, Expected=170.690 >Predicted=170.690, Expected=171.210 >Predicted=171.210, Expected=173.750 >Predicted=173.750, Expected=172.400 >Predicted=172.400, Expected=173.660 >Predicted=173.660, Expected=174.910 RMSE: 2.982
<ipython-input-7-c5f65b306fc0>:2: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True) # prepare data
# summary statistics of time series
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
print(series.describe())
count 947.000000 mean 138.030993 std 32.288810 min 56.092499 25% 121.779999 50% 143.779999 75% 161.985001 max 196.449997 Name: 1, dtype: float64
<ipython-input-8-3512b164cf88>:3: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
series.plot()
pyplot.show()
<ipython-input-9-95d605ddc644>:3: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
pyplot.figure(1)
pyplot.subplot(211)
series.hist()
pyplot.subplot(212)
series.plot(kind='kde')
pyplot.show()
<ipython-input-10-25c0a3841568>:1: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
# Load data
df = pd.read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True)
# Group the data by year
groups = df.groupby(pd.Grouper(freq='A'))
# Create a list to store data by year
years_data = []
# Populate the list with data grouped by year
for name, group in groups:
years_data.append(group.values.flatten())
# Plot boxplots for each year
plt.figure(figsize=(15, 8))
plt.boxplot(years_data, labels=range(df.index.year.min(), df.index.year.max() + 1))
plt.title('Boxplots of Closing Prices by Year')
plt.xlabel('Year')
plt.ylabel('Closing Price')
plt.xticks(rotation=45)
plt.tight_layout()
plt.show()
# create a differenced time series
def difference(dataset):
diff = list()
for i in range(1, len(dataset)):
value = dataset[i] - dataset[i - 1]
diff.append(value)
return Series(diff)
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
X = series.values
# difference data
stationary = difference(X)
stationary.index = series.index[1:]
# check if stationary
result = adfuller(stationary)
print('ADF Statistic: %f' % result[0])
print('p-value: %f' % result[1])
print('Critical Values:')
for key, value in result[4].items():
print('\t%s: %.3f' % (key, value)) # save
stationary.to_csv('/content/drive/MyDrive/data/app_stationary.csv', header=False)
<ipython-input-13-3f7766769d09>:9: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
ADF Statistic: -32.042208 p-value: 0.000000 Critical Values: 1%: -3.437 5%: -2.865 10%: -2.568
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
pyplot.figure()
pyplot.subplot(211)
plot_acf(series, lags=50, ax=pyplot.gca())
pyplot.subplot(212)
plot_pacf(series, lags=50, ax=pyplot.gca())
pyplot.show()
<ipython-input-14-4a966e6d86af>:1: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
# load data
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
# prepare data
X = series.values
X = X.astype('float32')
train_size = int(len(X) * 0.50)
train, test = X[0:train_size], X[train_size:]
# walk-forward validation
history = [x for x in train]
predictions = list()
for i in range(len(test)):
# predict
model = ARIMA(history, order=(0,1,2))
model_fit = model.fit()
yhat = model_fit.forecast()[0]
predictions.append(yhat)
# observation
obs = test[i]
history.append(obs)
print('>Predicted=%.3f, Expected=%.3f' % (yhat, obs))
# report performance
rmse = sqrt(mean_squared_error(test, predictions))
print('RMSE: %.3f' % rmse)
<ipython-input-15-4218bb00964a>:2: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
>Predicted=150.052, Expected=174.910 RMSE: 19.768
# evaluate an ARIMA model for a given order (p,d,q) and return RMSE
def evaluate_arima_model(X, arima_order):
# prepare training dataset
X = X.astype('float32')
train_size = int(len(X) * 0.50)
train, test = X[0:train_size], X[train_size:]
history = [x for x in train]
# make predictions
predictions = list()
for t in range(len(test)):
model = ARIMA(history, order=arima_order)
model_fit = model.fit()
yhat = model_fit.forecast()[0]
predictions.append(yhat)
history.append(test[t])
# calculate out of sample error
rmse = sqrt(mean_squared_error(test, predictions))
return rmse
# evaluate combinations of p, d and q values for an ARIMA model
def evaluate_models(dataset, p_values, d_values, q_values):
dataset = dataset.astype('float32')
best_score, best_cfg = float("inf"), None
for p in p_values:
for d in d_values:
for q in q_values:
order = (p,d,q)
try:
rmse = evaluate_arima_model(dataset, order)
if rmse < best_score:
best_score, best_cfg = rmse, order
print('ARIMA%s RMSE=%.3f' % (order, rmse))
except:
continue
print('Best ARIMA%s RMSE=%.3f' % (best_cfg, best_score))
# load dataset
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
# evaluate parameters
p_values = range(0,2)
d_values = range(0, 3)
q_values = range(0, 2)
warnings.filterwarnings("ignore")
evaluate_models(series.values, p_values, d_values, q_values)
<ipython-input-5-589e6721015a>:40: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
ARIMA(0, 0, 0) RMSE=36.777 ARIMA(0, 0, 1) RMSE=19.132 ARIMA(0, 1, 0) RMSE=2.982 ARIMA(0, 1, 1) RMSE=2.995 ARIMA(0, 2, 0) RMSE=4.195 ARIMA(0, 2, 1) RMSE=2.985 ARIMA(1, 0, 0) RMSE=2.984 ARIMA(1, 0, 1) RMSE=2.996 ARIMA(1, 1, 0) RMSE=2.996 ARIMA(1, 1, 1) RMSE=2.999 ARIMA(1, 2, 0) RMSE=3.747 ARIMA(1, 2, 1) RMSE=3.000 Best ARIMA(0, 1, 0) RMSE=2.982
# plot residual errors for ARIMA model
# load data
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
# prepare data
X = series.values
X = X.astype('float32')
train_size = int(len(X) * 0.50)
train, test = X[0:train_size], X[train_size:]
# walk-forward validation
history = [x for x in train]
predictions = list()
for i in range(len(test)):
# predict
model = ARIMA(history, order=(0,1,0))
model_fit = model.fit()
yhat = model_fit.forecast()[0]
predictions.append(yhat)
# observation
obs = test[i]
history.append(obs)
# errors
residuals = [test[i]-predictions[i] for i in range(len(test))]
residuals = DataFrame(residuals)
pyplot.figure()
pyplot.subplot(211)
residuals.hist(ax=pyplot.gca())
pyplot.subplot(212)
residuals.plot(kind='kde', ax=pyplot.gca())
pyplot.show()
<ipython-input-3-c8f3bfbe71cd>:8: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
# ACF and PACF plots of forecast residual errors
# load data
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
# prepare data
X = series.values
X = X.astype('float32')
train_size = int(len(X) * 0.50)
train, test = X[0:train_size], X[train_size:]
# walk-forward validation
history = [x for x in train]
predictions = list()
for i in range(len(test)):
# predict
model = ARIMA(history, order=(0,1,0))
model_fit = model.fit()
yhat = model_fit.forecast()[0]
predictions.append(yhat)
# observation
obs = test[i]
history.append(obs)
# errors
residuals = [test[i]-predictions[i] for i in range(len(test))]
residuals = DataFrame(residuals)
pyplot.figure()
pyplot.subplot(211)
plot_acf(residuals, lags=25, ax=pyplot.gca())
pyplot.subplot(212)
plot_pacf(residuals, lags=25, ax=pyplot.gca())
pyplot.show()
<ipython-input-4-6f3566148926>:9: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
# plots of box-cox transformed dataset
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
X = series.values
transformed, lam = boxcox(X)
print('Lambda: %f' % lam)
pyplot.figure(1)
# line plot
pyplot.subplot(311)
pyplot.plot(transformed)
# histogram
pyplot.subplot(312)
pyplot.hist(transformed)
# q-q plot
pyplot.subplot(313)
qqplot(transformed, line='r', ax=pyplot.gca())
pyplot.show()
<ipython-input-5-dfa7deb9ef96>:6: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
Lambda: 1.953080
# Function to invert Box-Cox transform
def boxcox_inverse(value, lam):
if lam == 0:
return exp(value)
return exp(log(lam * value + 1) / lam)
# Load data
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
# Prepare data
X = series.values
X = X.astype('float32')
train_size = int(len(X) * 0.50)
train, test = X[0:train_size], X[train_size:]
# Walk-forward validation
history = [x for x in train]
predictions = list()
for i in range(len(test)):
# Transform
transformed, lam = boxcox(history)
if lam < -5:
transformed, lam = history, 1
# Predict
model = ARIMA(transformed, order=(0,1,0))
model_fit = model.fit()
yhat = model_fit.forecast()[0]
# Invert transformed prediction
yhat = boxcox_inverse(yhat, lam)
predictions.append(yhat)
# Observation
obs = test[i]
history.append(obs)
print('>Predicted=%.3f, Expected=%.3f' % (yhat, obs))
# Report performance
rmse = sqrt(mean_squared_error(test, predictions))
print('RMSE: %.3f' % rmse)
<ipython-input-6-1d79d03f6b50>:16: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
>Predicted=150.000, Expected=151.000 >Predicted=151.000, Expected=153.490 >Predicted=153.490, Expected=157.870 >Predicted=157.870, Expected=160.550 >Predicted=160.550, Expected=161.020 >Predicted=161.020, Expected=161.410 >Predicted=161.410, Expected=161.940 >Predicted=161.940, Expected=156.810 >Predicted=156.810, Expected=160.240 >Predicted=160.240, Expected=165.300 >Predicted=165.300, Expected=164.770 >Predicted=164.770, Expected=163.760 >Predicted=163.760, Expected=161.840 >Predicted=161.840, Expected=165.320 >Predicted=165.320, Expected=171.180 >Predicted=171.180, Expected=175.080 >Predicted=175.080, Expected=174.560 >Predicted=174.560, Expected=179.450 >Predicted=179.450, Expected=175.740 >Predicted=175.740, Expected=174.330 >Predicted=174.330, Expected=179.300 >Predicted=179.300, Expected=172.260 >Predicted=172.260, Expected=171.140 >Predicted=171.140, Expected=169.750 >Predicted=169.750, Expected=172.990 >Predicted=172.990, Expected=175.640 >Predicted=175.640, Expected=176.280 >Predicted=176.280, Expected=180.330 >Predicted=180.330, Expected=179.290 >Predicted=179.290, Expected=179.380 >Predicted=179.380, Expected=178.200 >Predicted=178.200, Expected=177.570 >Predicted=177.570, Expected=182.010 >Predicted=182.010, Expected=179.700 >Predicted=179.700, Expected=174.920 >Predicted=174.920, Expected=172.000 >Predicted=172.000, Expected=172.170 >Predicted=172.170, Expected=172.190 >Predicted=172.190, Expected=175.080 >Predicted=175.080, Expected=175.530 >Predicted=175.530, Expected=172.190 >Predicted=172.190, Expected=173.070 >Predicted=173.070, Expected=169.800 >Predicted=169.800, Expected=166.230 >Predicted=166.230, Expected=164.510 >Predicted=164.510, Expected=162.410 >Predicted=162.410, Expected=161.620 >Predicted=161.620, Expected=159.780 >Predicted=159.780, Expected=159.690 >Predicted=159.690, Expected=159.220 >Predicted=159.220, Expected=170.330 >Predicted=170.330, Expected=174.780 >Predicted=174.780, Expected=174.610 >Predicted=174.610, Expected=175.840 >Predicted=175.840, Expected=172.900 >Predicted=172.900, Expected=172.390 >Predicted=172.390, Expected=171.660 >Predicted=171.660, Expected=174.830 >Predicted=174.830, Expected=176.280 >Predicted=176.280, Expected=172.120 >Predicted=172.120, Expected=168.640 >Predicted=168.640, Expected=168.880 >Predicted=168.880, Expected=172.790 >Predicted=172.790, Expected=172.550 >Predicted=172.550, Expected=168.880 >Predicted=168.880, Expected=167.300 >Predicted=167.300, Expected=164.320 >Predicted=164.320, Expected=160.070 >Predicted=160.070, Expected=162.740 >Predicted=162.740, Expected=164.850 >Predicted=164.850, Expected=165.120 >Predicted=165.120, Expected=163.200 >Predicted=163.200, Expected=166.560 >Predicted=166.560, Expected=166.230 >Predicted=166.230, Expected=163.170 >Predicted=163.170, Expected=159.300 >Predicted=159.300, Expected=157.440 >Predicted=157.440, Expected=162.950 >Predicted=162.950, Expected=158.520 >Predicted=158.520, Expected=154.730 >Predicted=154.730, Expected=150.620 >Predicted=150.620, Expected=155.090 >Predicted=155.090, Expected=159.590 >Predicted=159.590, Expected=160.620 >Predicted=160.620, Expected=163.980 >Predicted=163.980, Expected=165.380 >Predicted=165.380, Expected=168.820 >Predicted=168.820, Expected=170.210 >Predicted=170.210, Expected=174.070 >Predicted=174.070, Expected=174.720 >Predicted=174.720, Expected=175.600 >Predicted=175.600, Expected=178.960 >Predicted=178.960, Expected=177.770 >Predicted=177.770, Expected=174.610 >Predicted=174.610, Expected=174.310 >Predicted=174.310, Expected=178.440 >Predicted=178.440, Expected=175.060 >Predicted=175.060, Expected=171.830 >Predicted=171.830, Expected=172.140 >Predicted=172.140, Expected=170.090 >Predicted=170.090, Expected=165.750 >Predicted=165.750, Expected=167.660 >Predicted=167.660, Expected=170.400 >Predicted=170.400, Expected=165.290 >Predicted=165.290, Expected=165.070 >Predicted=165.070, Expected=167.400 >Predicted=167.400, Expected=167.230 >Predicted=167.230, Expected=166.420 >Predicted=166.420, Expected=161.790 >Predicted=161.790, Expected=162.880 >Predicted=162.880, Expected=156.800 >Predicted=156.800, Expected=156.570 >Predicted=156.570, Expected=163.640 >Predicted=163.640, Expected=157.650 >Predicted=157.650, Expected=157.960 >Predicted=157.960, Expected=159.480 >Predicted=159.480, Expected=166.020 >Predicted=166.020, Expected=156.770 >Predicted=156.770, Expected=157.280 >Predicted=157.280, Expected=152.060 >Predicted=152.060, Expected=154.510 >Predicted=154.510, Expected=146.500 >Predicted=146.500, Expected=142.560 >Predicted=142.560, Expected=147.110 >Predicted=147.110, Expected=145.540 >Predicted=145.540, Expected=149.240 >Predicted=149.240, Expected=140.820 >Predicted=140.820, Expected=137.350 >Predicted=137.350, Expected=137.590 >Predicted=137.590, Expected=143.110 >Predicted=143.110, Expected=140.360 >Predicted=140.360, Expected=140.520 >Predicted=140.520, Expected=143.780 >Predicted=143.780, Expected=149.640 >Predicted=149.640, Expected=148.840 >Predicted=148.840, Expected=148.710 >Predicted=148.710, Expected=151.210 >Predicted=151.210, Expected=145.380 >Predicted=145.380, Expected=146.140 >Predicted=146.140, Expected=148.710 >Predicted=148.710, Expected=147.960 >Predicted=147.960, Expected=142.640 >Predicted=142.640, Expected=137.130 >Predicted=137.130, Expected=131.880 >Predicted=131.880, Expected=132.760 >Predicted=132.760, Expected=135.430 >Predicted=135.430, Expected=130.060 >Predicted=130.060, Expected=131.560 >Predicted=131.560, Expected=135.870 >Predicted=135.870, Expected=135.350 >Predicted=135.350, Expected=138.270 >Predicted=138.270, Expected=141.660 >Predicted=141.660, Expected=141.660 >Predicted=141.660, Expected=137.440 >Predicted=137.440, Expected=139.230 >Predicted=139.230, Expected=136.720 >Predicted=136.720, Expected=138.930 >Predicted=138.930, Expected=141.560 >Predicted=141.560, Expected=142.920 >Predicted=142.920, Expected=146.350 >Predicted=146.350, Expected=147.040 >Predicted=147.040, Expected=144.870 >Predicted=144.870, Expected=145.860 >Predicted=145.860, Expected=145.490 >Predicted=145.490, Expected=148.470 >Predicted=148.470, Expected=150.170 >Predicted=150.170, Expected=147.070 >Predicted=147.070, Expected=151.000 >Predicted=151.000, Expected=153.040 >Predicted=153.040, Expected=155.350 >Predicted=155.350, Expected=154.090 >Predicted=154.090, Expected=152.950 >Predicted=152.950, Expected=151.600 >Predicted=151.600, Expected=156.790 >Predicted=156.790, Expected=157.350 >Predicted=157.350, Expected=162.510 >Predicted=162.510, Expected=161.510 >Predicted=161.510, Expected=160.010 >Predicted=160.010, Expected=166.130 >Predicted=166.130, Expected=165.810 >Predicted=165.810, Expected=165.350 >Predicted=165.350, Expected=164.870 >Predicted=164.870, Expected=164.920 >Predicted=164.920, Expected=169.240 >Predicted=169.240, Expected=168.490 >Predicted=168.490, Expected=172.100 >Predicted=172.100, Expected=173.190 >Predicted=173.190, Expected=173.030 >Predicted=173.030, Expected=174.550 >Predicted=174.550, Expected=174.150 >Predicted=174.150, Expected=171.520 >Predicted=171.520, Expected=167.570 >Predicted=167.570, Expected=167.230 >Predicted=167.230, Expected=167.530 >Predicted=167.530, Expected=170.030 >Predicted=170.030, Expected=163.620 >Predicted=163.620, Expected=161.380 >Predicted=161.380, Expected=158.910 >Predicted=158.910, Expected=157.220 >Predicted=157.220, Expected=157.960 >Predicted=157.960, Expected=155.810 >Predicted=155.810, Expected=154.530 >Predicted=154.530, Expected=155.960 >Predicted=155.960, Expected=154.460 >Predicted=154.460, Expected=157.370 >Predicted=157.370, Expected=163.430 >Predicted=163.430, Expected=153.840 >Predicted=153.840, Expected=155.310 >Predicted=155.310, Expected=152.370 >Predicted=152.370, Expected=150.700 >Predicted=150.700, Expected=154.480 >Predicted=154.480, Expected=156.900 >Predicted=156.900, Expected=153.720 >Predicted=153.720, Expected=152.740 >Predicted=152.740, Expected=150.430 >Predicted=150.430, Expected=150.770 >Predicted=150.770, Expected=151.760 >Predicted=151.760, Expected=149.840 >Predicted=149.840, Expected=142.480 >Predicted=142.480, Expected=138.200 >Predicted=138.200, Expected=142.450 >Predicted=142.450, Expected=146.100 >Predicted=146.100, Expected=146.400 >Predicted=146.400, Expected=145.430 >Predicted=145.430, Expected=140.090 >Predicted=140.090, Expected=140.420 >Predicted=140.420, Expected=138.980 >Predicted=138.980, Expected=138.340 >Predicted=138.340, Expected=142.990 >Predicted=142.990, Expected=138.380 >Predicted=138.380, Expected=142.410 >Predicted=142.410, Expected=143.750 >Predicted=143.750, Expected=143.860 >Predicted=143.860, Expected=143.390 >Predicted=143.390, Expected=147.270 >Predicted=147.270, Expected=149.450 >Predicted=149.450, Expected=152.340 >Predicted=152.340, Expected=149.350 >Predicted=149.350, Expected=144.800 >Predicted=144.800, Expected=155.740 >Predicted=155.740, Expected=153.340 >Predicted=153.340, Expected=150.650 >Predicted=150.650, Expected=145.030 >Predicted=145.030, Expected=138.880 >Predicted=138.880, Expected=138.380 >Predicted=138.380, Expected=138.920 >Predicted=138.920, Expected=139.500 >Predicted=139.500, Expected=134.870 >Predicted=134.870, Expected=146.870 >Predicted=146.870, Expected=149.700 >Predicted=149.700, Expected=148.280 >Predicted=148.280, Expected=150.040 >Predicted=150.040, Expected=148.790 >Predicted=148.790, Expected=150.720 >Predicted=150.720, Expected=151.290 >Predicted=151.290, Expected=148.010 >Predicted=148.010, Expected=150.180 >Predicted=150.180, Expected=151.070 >Predicted=151.070, Expected=148.110 >Predicted=148.110, Expected=144.220 >Predicted=144.220, Expected=141.170 >Predicted=141.170, Expected=148.030 >Predicted=148.030, Expected=148.310 >Predicted=148.310, Expected=147.810 >Predicted=147.810, Expected=146.630 >Predicted=146.630, Expected=142.910 >Predicted=142.910, Expected=140.940 >Predicted=140.940, Expected=142.650 >Predicted=142.650, Expected=142.160 >Predicted=142.160, Expected=144.490 >Predicted=144.490, Expected=145.470 >Predicted=145.470, Expected=143.210 >Predicted=143.210, Expected=136.500 >Predicted=136.500, Expected=134.510 >Predicted=134.510, Expected=132.370 >Predicted=132.370, Expected=132.300 >Predicted=132.300, Expected=135.450 >Predicted=135.450, Expected=132.230 >Predicted=132.230, Expected=131.860 >Predicted=131.860, Expected=130.030 >Predicted=130.030, Expected=126.040 >Predicted=126.040, Expected=129.610 >Predicted=129.610, Expected=129.930 >Predicted=129.930, Expected=125.070 >Predicted=125.070, Expected=126.360 >Predicted=126.360, Expected=125.020 >Predicted=125.020, Expected=129.620 >Predicted=129.620, Expected=130.150 >Predicted=130.150, Expected=130.730 >Predicted=130.730, Expected=133.490 >Predicted=133.490, Expected=133.410 >Predicted=133.410, Expected=134.760 >Predicted=134.760, Expected=135.940 >Predicted=135.940, Expected=135.210 >Predicted=135.210, Expected=135.270 >Predicted=135.270, Expected=137.870 >Predicted=137.870, Expected=141.110 >Predicted=141.110, Expected=142.530 >Predicted=142.530, Expected=141.860 >Predicted=141.860, Expected=143.960 >Predicted=143.960, Expected=145.930 >Predicted=145.930, Expected=143.000 >Predicted=143.000, Expected=144.290 >Predicted=144.290, Expected=145.430 >Predicted=145.430, Expected=150.820 >Predicted=150.820, Expected=154.500 >Predicted=154.500, Expected=151.730 >Predicted=151.730, Expected=154.650 >Predicted=154.650, Expected=151.920 >Predicted=151.920, Expected=150.870 >Predicted=150.870, Expected=151.010 >Predicted=151.010, Expected=153.850 >Predicted=153.850, Expected=153.200 >Predicted=153.200, Expected=155.330 >Predicted=155.330, Expected=153.710 >Predicted=153.710, Expected=152.550 >Predicted=152.550, Expected=148.480 >Predicted=148.480, Expected=148.910 >Predicted=148.910, Expected=149.400 >Predicted=149.400, Expected=146.710 >Predicted=146.710, Expected=147.920 >Predicted=147.920, Expected=147.410 >Predicted=147.410, Expected=145.310 >Predicted=145.310, Expected=145.910 >Predicted=145.910, Expected=151.030 >Predicted=151.030, Expected=153.830 >Predicted=153.830, Expected=151.600 >Predicted=151.600, Expected=152.870 >Predicted=152.870, Expected=150.590 >Predicted=150.590, Expected=148.500 >Predicted=148.500, Expected=150.470 >Predicted=150.470, Expected=152.590 >Predicted=152.590, Expected=152.990 >Predicted=152.990, Expected=155.850 >Predicted=155.850, Expected=155.000 >Predicted=155.000, Expected=157.400 >Predicted=157.400, Expected=159.280 >Predicted=159.280, Expected=157.830 >Predicted=157.830, Expected=158.930 >Predicted=158.930, Expected=160.250 >Predicted=160.250, Expected=158.280 >Predicted=158.280, Expected=157.650 >Predicted=157.650, Expected=160.770 >Predicted=160.770, Expected=162.360 >Predicted=162.360, Expected=164.900 >Predicted=164.900, Expected=166.170 >Predicted=166.170, Expected=165.630 >Predicted=165.630, Expected=163.760 >Predicted=163.760, Expected=164.660 >Predicted=164.660, Expected=162.030 >Predicted=162.030, Expected=160.800 >Predicted=160.800, Expected=160.100 >Predicted=160.100, Expected=165.560 >Predicted=165.560, Expected=165.210 >Predicted=165.210, Expected=165.230 >Predicted=165.230, Expected=166.470 >Predicted=166.470, Expected=167.630 >Predicted=167.630, Expected=166.650 >Predicted=166.650, Expected=165.020 >Predicted=165.020, Expected=165.330 >Predicted=165.330, Expected=163.770 >Predicted=163.770, Expected=163.760 >Predicted=163.760, Expected=168.410 >Predicted=168.410, Expected=169.680 >Predicted=169.680, Expected=169.590 >Predicted=169.590, Expected=168.540 >Predicted=168.540, Expected=167.450 >Predicted=167.450, Expected=165.790 >Predicted=165.790, Expected=173.570 >Predicted=173.570, Expected=173.500 >Predicted=173.500, Expected=171.770 >Predicted=171.770, Expected=173.560 >Predicted=173.560, Expected=173.750 >Predicted=173.750, Expected=172.570 >Predicted=172.570, Expected=172.070 >Predicted=172.070, Expected=172.070 >Predicted=172.070, Expected=172.690 >Predicted=172.690, Expected=175.050 >Predicted=175.050, Expected=175.160 >Predicted=175.160, Expected=174.200 >Predicted=174.200, Expected=171.560 >Predicted=171.560, Expected=171.840 >Predicted=171.840, Expected=172.990 >Predicted=172.990, Expected=175.430 >Predicted=175.430, Expected=177.300 >Predicted=177.300, Expected=177.250 >Predicted=177.250, Expected=180.090 >Predicted=180.090, Expected=180.950 >Predicted=180.950, Expected=179.580 >Predicted=179.580, Expected=179.210 >Predicted=179.210, Expected=177.820 >Predicted=177.820, Expected=180.570 >Predicted=180.570, Expected=180.960 >Predicted=180.960, Expected=183.790 >Predicted=183.790, Expected=183.310 >Predicted=183.310, Expected=183.950 >Predicted=183.950, Expected=186.010 >Predicted=186.010, Expected=184.920 >Predicted=184.920, Expected=185.010 >Predicted=185.010, Expected=183.960 >Predicted=183.960, Expected=187.000 >Predicted=187.000, Expected=186.680 >Predicted=186.680, Expected=185.270 >Predicted=185.270, Expected=188.060 >Predicted=188.060, Expected=189.250 >Predicted=189.250, Expected=189.590 >Predicted=189.590, Expected=193.970 >Predicted=193.970, Expected=192.460 >Predicted=192.460, Expected=191.330 >Predicted=191.330, Expected=191.810 >Predicted=191.810, Expected=190.680 >Predicted=190.680, Expected=188.610 >Predicted=188.610, Expected=188.080 >Predicted=188.080, Expected=189.770 >Predicted=189.770, Expected=190.540 >Predicted=190.540, Expected=190.690 >Predicted=190.690, Expected=193.990 >Predicted=193.990, Expected=193.730 >Predicted=193.730, Expected=195.100 >Predicted=195.100, Expected=193.130 >Predicted=193.130, Expected=191.940 >Predicted=191.940, Expected=192.750 >Predicted=192.750, Expected=193.620 >Predicted=193.620, Expected=194.500 >Predicted=194.500, Expected=193.220 >Predicted=193.220, Expected=195.830 >Predicted=195.830, Expected=196.450 >Predicted=196.450, Expected=195.610 >Predicted=195.610, Expected=192.580 >Predicted=192.580, Expected=191.170 >Predicted=191.170, Expected=181.990 >Predicted=181.990, Expected=178.850 >Predicted=178.850, Expected=179.800 >Predicted=179.800, Expected=178.190 >Predicted=178.190, Expected=177.970 >Predicted=177.970, Expected=177.790 >Predicted=177.790, Expected=179.460 >Predicted=179.460, Expected=177.450 >Predicted=177.450, Expected=176.570 >Predicted=176.570, Expected=174.000 >Predicted=174.000, Expected=174.490 >Predicted=174.490, Expected=175.840 >Predicted=175.840, Expected=177.230 >Predicted=177.230, Expected=181.120 >Predicted=181.120, Expected=176.380 >Predicted=176.380, Expected=178.610 >Predicted=178.610, Expected=180.190 >Predicted=180.190, Expected=184.120 >Predicted=184.120, Expected=187.650 >Predicted=187.650, Expected=187.870 >Predicted=187.870, Expected=189.460 >Predicted=189.460, Expected=189.700 >Predicted=189.700, Expected=182.910 >Predicted=182.910, Expected=177.560 >Predicted=177.560, Expected=178.180 >Predicted=178.180, Expected=179.360 >Predicted=179.360, Expected=176.300 >Predicted=176.300, Expected=174.210 >Predicted=174.210, Expected=175.740 >Predicted=175.740, Expected=175.010 >Predicted=175.010, Expected=177.970 >Predicted=177.970, Expected=179.070 >Predicted=179.070, Expected=175.490 >Predicted=175.490, Expected=173.930 >Predicted=173.930, Expected=174.790 >Predicted=174.790, Expected=176.080 >Predicted=176.080, Expected=171.960 >Predicted=171.960, Expected=170.430 >Predicted=170.430, Expected=170.690 >Predicted=170.690, Expected=171.210 >Predicted=171.210, Expected=173.750 >Predicted=173.750, Expected=172.400 >Predicted=172.400, Expected=173.660 >Predicted=173.660, Expected=174.910 RMSE: 2.982
# finalize model and save to file
# load data
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
# prepare data
X = series.values
X = X.astype('float32')
# transform data
transformed, lam = boxcox(X)
# fit model
model = ARIMA(transformed, order=(0,1,0))
model_fit = model.fit()
# save model
model_fit.save('/content/drive/MyDrive/data/app_model.pkl')
numpy.save('/content/drive/MyDrive/data/app_model_lambda.npy', [lam])
<ipython-input-7-1cc29c08cf7c>:7: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
# load the finalized model and make a prediction
# invert box-cox transform
def boxcox_inverse(value, lam):
if lam == 0:
return exp(value)
return exp(log(lam * value + 1) / lam)
model_fit = ARIMAResults.load('/content/drive/MyDrive/data/app_model.pkl')
lam = numpy.load('/content/drive/MyDrive/data/app_model_lambda.npy')
yhat = model_fit.forecast()[0]
yhat = boxcox_inverse(yhat, lam)
print('Predicted: %.3f' % yhat)
Predicted: 174.910
<ipython-input-8-8fed440f2b22>:11: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.) return exp(log(lam * value + 1) / lam)
# invert box-cox transform
def boxcox_inverse(value, lam):
if lam == 0:
return exp(value)
return exp(log(lam * value + 1) / lam)
# load and prepare datasets
dataset = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
X = dataset.values.astype('float32')
history = [x for x in X]
validation = read_csv('/content/drive/MyDrive/data/app_validation.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
y = validation.values.astype('float32')
# load model
model_fit = ARIMAResults.load('/content/drive/MyDrive/data/app_model.pkl')
lam = numpy.load('/content/drive/MyDrive/data/app_model_lambda.npy')
# make first prediction
predictions = list()
yhat = model_fit.forecast()[0]
yhat = boxcox_inverse(yhat, lam)
predictions.append(yhat)
history.append(y[0])
print('>Predicted=%.3f, Expected=%.3f' % (yhat, y[0]))
# rolling forecasts
for i in range(1, len(y)):
# transform
transformed, lam = boxcox(history)
if lam < -5:
transformed, lam = history, 1
# predict
model = ARIMA(transformed, order=(0,1,0))
model_fit = model.fit()
yhat = model_fit.forecast()[0]
# invert transformed prediction
yhat = boxcox_inverse(yhat, lam)
predictions.append(yhat)
# observation
obs = y[i]
history.append(obs)
print('>Predicted=%.3f, Expected=%.3f' % (yhat, obs))
# report performance
rmse = sqrt(mean_squared_error(y, predictions))
print('RMSE: %.3f' % rmse)
pyplot.plot(y)
pyplot.plot(predictions, color='red')
pyplot.show()
<ipython-input-9-89857cb596e9>:19: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
dataset = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
<ipython-input-9-89857cb596e9>:22: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.
validation = read_csv('/content/drive/MyDrive/data/app_validation.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
<ipython-input-9-89857cb596e9>:16: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
return exp(log(lam * value + 1) / lam)
>Predicted=174.910, Expected=177.490 >Predicted=177.490, Expected=178.990 >Predicted=178.990, Expected=178.390 >Predicted=178.390, Expected=179.800 >Predicted=179.800, Expected=180.710 >Predicted=180.710, Expected=178.850 >Predicted=178.850, Expected=178.720 >Predicted=178.720, Expected=177.150 >Predicted=177.150, Expected=175.840 >Predicted=175.840, Expected=175.460 >Predicted=175.460, Expected=172.880 >Predicted=172.880, Expected=173.000 >Predicted=173.000, Expected=173.440 >Predicted=173.440, Expected=171.100 >Predicted=171.100, Expected=166.890 >Predicted=166.890, Expected=168.220 >Predicted=168.220, Expected=170.290 >Predicted=170.290, Expected=170.770 >Predicted=170.770, Expected=173.970 >Predicted=173.970, Expected=177.570 >Predicted=177.570, Expected=176.650 >Predicted=176.650, Expected=179.230 >Predicted=179.230, Expected=181.820 >Predicted=181.820, Expected=182.890 >Predicted=182.890, Expected=182.410 >Predicted=182.410, Expected=186.400 >Predicted=186.400, Expected=184.800
/usr/local/lib/python3.10/dist-packages/numpy/core/_methods.py:187: RuntimeWarning: overflow encountered in reduce ret = umr_sum(x, axis, dtype, out, keepdims=keepdims, where=where)
>Predicted=184.800, Expected=187.440 >Predicted=187.440, Expected=188.010 >Predicted=188.010, Expected=189.710 >Predicted=189.710, Expected=189.690 >Predicted=189.690, Expected=191.450 >Predicted=191.450, Expected=190.640 >Predicted=190.640, Expected=191.310 >Predicted=191.310, Expected=189.970 >Predicted=189.970, Expected=189.790 >Predicted=189.790, Expected=190.400 >Predicted=190.400, Expected=189.370 >Predicted=189.370, Expected=189.950 >Predicted=189.950, Expected=191.240 >Predicted=191.240, Expected=189.430 >Predicted=189.430, Expected=193.420 >Predicted=193.420, Expected=192.320 >Predicted=192.320, Expected=194.270 >Predicted=194.270, Expected=195.710 >Predicted=195.710, Expected=193.180 >Predicted=193.180, Expected=194.710 >Predicted=194.710, Expected=197.960 >Predicted=197.960, Expected=198.110 >Predicted=198.110, Expected=197.570 >Predicted=197.570, Expected=195.890 >Predicted=195.890, Expected=196.940 >Predicted=196.940, Expected=194.830 >Predicted=194.830, Expected=194.680 >Predicted=194.680, Expected=193.600 >Predicted=193.600, Expected=193.050 >Predicted=193.050, Expected=193.150 >Predicted=193.150, Expected=193.580 >Predicted=193.580, Expected=192.530 >Predicted=192.530, Expected=185.640 >Predicted=185.640, Expected=184.250 >Predicted=184.250, Expected=181.910 >Predicted=181.910, Expected=181.180 >Predicted=181.180, Expected=185.560 >Predicted=185.560, Expected=185.140 >Predicted=185.140, Expected=186.190 >Predicted=186.190, Expected=185.590 >Predicted=185.590, Expected=185.920 >Predicted=185.920, Expected=183.630 >Predicted=183.630, Expected=182.680 >Predicted=182.680, Expected=188.630 >Predicted=188.630, Expected=191.560 >Predicted=191.560, Expected=193.890 >Predicted=193.890, Expected=195.180 >Predicted=195.180, Expected=194.500 >Predicted=194.500, Expected=194.170 >Predicted=194.170, Expected=192.420 >Predicted=192.420, Expected=191.730 >Predicted=191.730, Expected=188.040 >Predicted=188.040, Expected=184.400 >Predicted=184.400, Expected=186.860 >Predicted=186.860, Expected=185.850 >Predicted=185.850, Expected=187.680 >Predicted=187.680, Expected=189.300 >Predicted=189.300, Expected=189.410 >Predicted=189.410, Expected=188.320 >Predicted=188.320, Expected=188.850 >Predicted=188.850, Expected=187.150 >Predicted=187.150, Expected=185.040 >Predicted=185.040, Expected=184.150 >Predicted=184.150, Expected=183.860 >Predicted=183.860, Expected=182.310 >Predicted=182.310, Expected=181.560 >Predicted=181.560, Expected=182.320 >Predicted=182.320, Expected=184.370 >Predicted=184.370, Expected=182.520 >Predicted=182.520, Expected=181.160 >Predicted=181.160, Expected=182.630 >Predicted=182.630, Expected=181.420 >Predicted=181.420, Expected=180.750 RMSE: 1.986
| Name | Tasks | Time Spent |
|---|---|---|
| Hemraj Yadav | Load Data, Vizualization, EDA, Decomposition, TS Model, Prediction | 40 hours |
| Amit Sai Jitta | Time Series Decomposition, TS Model, Prediction | 25 hours |
More information on the fasteda package.
Yahoo Finance: https://en.wikipedia.org/wiki/List_of_S%26P_500_companies
https://www.kaggle.com/code/faressayah/stock-market-analysis-prediction-using-lstm
https://plotly.com/python/templates/
Coding practices from class mainly 10 and 11
from google.colab import drive
import nbconvert
# Mount Google Drive to access files
drive.mount('/content/drive')
# Provide the path to your notebook file
notebook_path = '/content/drive/MyDrive/Colab Notebooks/Group11_Final_Project_Part_2_TMSP_V2.ipynb' # Replace with your actual notebook path
# Convert notebook to HTML
html_exporter = nbconvert.HTMLExporter()
(body, resources) = html_exporter.from_filename(notebook_path)
# Write HTML content to a file
html_file_path = '/content/drive/MyDrive/Colab Notebooks/Group11_Final_Project_Part_2_TMSP_V2.html' # Replace with your desired HTML file path
with open(html_file_path, 'w') as f:
f.write(body)